-- Logs begin at Mon 2022-07-11 20:12:39 UTC, end at Mon 2022-07-11 21:21:42 UTC. -- Jul 11 20:43:01.184993 dv-cinder-849022-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 11 20:43:01.192236 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 11 20:43:01 2022] *** Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: nodename: dv-cinder-849022-2-iscsi Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: machine: x86_64 Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: clock source: unix Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: pcre jit disabled Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: detected number of CPU cores: 4 Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: current working directory: / Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: detected binary path: /usr/bin/uwsgi-core Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: your processes number limit is 47680 Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: your memory page size is 4096 bytes Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: detected max file descriptor number: 2048 Jul 11 20:43:01.194240 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: lock engine: pthread robust mutexes Jul 11 20:43:01.195104 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: thunder lock: enabled Jul 11 20:43:01.195104 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 11 20:43:01.195104 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 11 20:43:01.239600 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: Python main interpreter initialized at 0x564f2ddf49c0 Jul 11 20:43:01.239600 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: python threads support enabled Jul 11 20:43:01.239600 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: your server socket listen backlog is limited to 100 connections Jul 11 20:43:01.239600 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: your mercy for graceful operations on workers is 90 seconds Jul 11 20:43:01.239991 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: mapped 403077 bytes (393 KB) for 2 cores Jul 11 20:43:01.240060 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: *** Operational MODE: preforking *** Jul 11 20:43:01.240357 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: *** uWSGI is running in multiple interpreter mode *** Jul 11 20:43:01.240357 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: spawned uWSGI master process (pid: 80301) Jul 11 20:43:01.240576 dv-cinder-849022-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 11 20:43:01.244158 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: spawned uWSGI worker 1 (pid: 80310, cores: 1) Jul 11 20:43:01.244158 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: spawned uWSGI worker 2 (pid: 80311, cores: 1) Jul 11 20:43:01.244158 dv-cinder-849022-2-iscsi devstack@keystone.service[80301]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 11 20:43:06.616722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 20:43:06.616722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: from cryptography.utils import int_from_bytes Jul 11 20:43:06.617549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 20:43:06.617549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: from cryptography.utils import int_from_bytes Jul 11 20:43:06.617948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 20:43:06.617948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: from cryptography.utils import int_from_bytes Jul 11 20:43:06.618057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 20:43:06.618057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: from cryptography.utils import int_from_bytes Jul 11 20:43:06.665487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 20:43:06.665684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 20:43:06.665684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 20:43:06.665826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 20:43:06.665894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] command line args: [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 20:43:06.665990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] command line args: [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 20:43:06.666129 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 20:43:06.666263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 20:43:06.666329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ================================================================================ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 20:43:06.666389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ================================================================================ {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 20:43:06.666708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] admin_token = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.666829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] admin_token = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.667023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] config_dir = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.667146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config_dir = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.667146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.667286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.667286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] config_source = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.667399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config_source = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.667470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] debug = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.667531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] debug = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.667690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.667907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.668007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.668069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.668232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.668316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.668493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.668620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.668695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.668761 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.668905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.668983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.669078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_config_append = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.669159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_config_append = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.669261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.669334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.669432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_dir = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.669510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_dir = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.669571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_file = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.669636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.669837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.669963 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.670031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.670089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.670279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.670399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.670526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.670609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.670679 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.670746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.670903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.671041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.671134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.671214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.671307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.671400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.671554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.671660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.671879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.672143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.672143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.672385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.672539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.672649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.672754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.672894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.673007 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.673114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.673254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.673371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.673478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.673622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.673739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] publish_errors = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.673876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] publish_errors = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.673983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.674090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.674231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.674336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.674442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.674596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.674707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.674864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.674982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] standard_threads = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] standard_threads = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] use_journal = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.675994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_journal = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.676066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] use_json = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.676153 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_json = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.676236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] use_stderr = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.676309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_stderr = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.676387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] use_syslog = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.676465 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_syslog = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.676572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.676653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 20:43:06.676730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.676804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.676882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.676954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.677029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.677107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.677267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.677396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.677544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.677665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.677787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.677936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.678057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.678172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.678293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.678413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.678531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.678644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.678755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.678874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.678998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.679117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.679231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.679348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.679466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.679579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.679701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.679813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.679927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.680038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.680177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.680292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.680527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.680678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.680801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.680916 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.681029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.681146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.681269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.681382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.681494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.681695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.681813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.681925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.682047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.682159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.682271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.682271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.682490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.682603 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.682724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.682837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.682959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.683072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.683182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.683293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.683414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.683527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.683638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.683805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.683921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.684047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.684183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.684296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.684408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.684531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.684651 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.684775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.684887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.684998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.685108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.685225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.685336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.685460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.685579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.685691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.685810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.685921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.686032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.686216 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.686330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.686440 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.686552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.686669 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.686786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.686911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.687025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.687135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.687252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.687363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.687478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.687598 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.687720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.687838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.687950 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.688061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.688186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.688315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.688429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.688526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.688626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.688722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.688816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.688921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.689034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.689131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.689225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.689319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.689444 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.689571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.689686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.689797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.689908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.690028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.690147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.690266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.690378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.690489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.690600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.690710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.690829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.690949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.691075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.691188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.691310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.691430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.691535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.691654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.691768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.691890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.692002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.692174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.692289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.692401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.692512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.692638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.692751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.692855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.692964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.693075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.693187 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.693302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.693414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.693525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.693627 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.693736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.693857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.693961 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.694071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.694188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.694300 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.694411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.694530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.694643 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.694760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.694879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.694992 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.695108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.695241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.695364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.695482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.695594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.695708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.695818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.695818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.696039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.696039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.696274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.696387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.696498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.696620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.696739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.696739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.696957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.697068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.697217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.697466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.697592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.697712 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.697831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.697943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.698056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.698176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.698298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.698414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.698518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.698630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.698742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.698854 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.698965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.699086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.699198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.699309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.699420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.699530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.699677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.699800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.699921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.700034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.700161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.700285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.700398 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.700518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.700630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.700741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.700852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.700963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.external = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.701074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.701198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.701317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.701429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.701541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.external = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.701651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.701799 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.701921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.password = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.702035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.702156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.token = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.702277 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.702431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.702548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.password = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.702662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.702783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.token = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.702896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.703008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.703129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.703242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.703354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.703824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.703945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.704057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.704212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.704327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.704439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.704565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.704680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.704792 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.704903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.705016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.705128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.705249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.705363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.705475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.705587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.705698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.705810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.705934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.706048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.706160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.706271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.706383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.706495 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.706616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.706729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.706841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.706953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.707064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.707185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.707309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.707423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.707541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.707661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.707781 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.707895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.708016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.708156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.708265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.708376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.708488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.708603 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.708724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.708836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.708948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.709060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.709172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.709284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.709284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.709505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.709618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.709730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.709843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.709954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.710075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.710189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.710300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.710412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.710530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.710645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.710765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.710878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.710990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.711102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.711220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.711340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.711527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.711642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.711754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.711866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.711978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.712100 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.712243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.712355 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.712467 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.712578 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.712690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.712814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.712961 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.713075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.713187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.713299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.713412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.713533 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.713647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.713760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.713873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.713986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.714098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.714219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.714332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.714445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.714557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.714671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.714783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.714905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.715026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.715141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.715253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.715366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.715478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.715599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.715713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.715825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.715938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.716051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.716179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.716305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.716429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.716544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.716657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.716770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.716882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.717004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.717117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.717230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.717342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.717458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.717571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.717766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.717885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.717999 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.718119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.718245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.718358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.718480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.718594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.718707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.718819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.718932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.719045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.719167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.719280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.719393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.719505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.719617 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.719729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.719857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.719972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.720086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.720254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.720367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.720479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.720606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.720721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.720834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.720947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.721059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.721172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.721293 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.721407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.721520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.721633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.721745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.721857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.721984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.722099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.722213 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.722326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.722438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.722550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.722673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.722787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.722900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.723011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.723125 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.723238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.723360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.723474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.723587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.723700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.723812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.723933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.724048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.724175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.724289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.724402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.724514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.724640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.724640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.724856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.724969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.725081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.725193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.725315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.725429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.725542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.725661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.725775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.725775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.725997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.726113 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.726225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.726339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.726452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.726566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.726687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.726687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.726900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.727012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.727125 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.727237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.727359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.727478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.727591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.727704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.727816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.727929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.728050 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.728222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.728336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.728450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.728563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.728676 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.728801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.728916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.729030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.729142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.729255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.729367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.729489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.729605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.729718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.729830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.729943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.730055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.730176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.730290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.730403 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.730515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.730628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.730741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.730865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.730980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.730980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.731191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.731303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.731415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.731535 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.731650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.731762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.731874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.731986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.732098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.732244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.732366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.732479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.732592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.732706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.732818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.732938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.733052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.733163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.733276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.733387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] role.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.733499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.733687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] role.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.733804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.733917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.734029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.734143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.734255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.734375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.734490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.734602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.734715 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.734827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.734939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.735059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.735174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.735286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.735398 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.735509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.735621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.735745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.735861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.735973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.736085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.736237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.736362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.736477 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.736589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.736701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.736814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.736926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.737048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.737048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.737259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.737259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.737474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.737618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.737741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.737741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.737954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.738066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.738178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.738178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.738397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.738511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.738623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.738734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.738846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.738959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.739082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.739196 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.739308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.739419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.739531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.739642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.739762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.739876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.739990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.740101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.740247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.740358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.740484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.740598 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.740711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.740824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.740936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.741057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.741170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.741282 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.741400 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.741512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.741512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.741731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.741845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.741956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.742070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.742182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.742293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.742414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.742414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.742632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.742745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.742857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.742969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.743089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.743204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.743315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.743427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.743539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.743539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.743757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.743871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.743982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.744230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.744387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.744502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.744631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.744746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.744859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.744971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.745083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.745196 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.745317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.745317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.745530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.745643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.745755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.connection = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.745877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.745999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.746113 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.746234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.746348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.746460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.746572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.746684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.746796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.746916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.747030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.747156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.747279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.747394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.747511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.747624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.747745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.747860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.747972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.748085 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.748245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.748359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.748485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.748601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.748713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.748825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.748937 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.749049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.749169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.749283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.749387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.749498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.749610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.749721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.749842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.749956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.749956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 20:43:06.750167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.750167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.750167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.750167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.750167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.750167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 20:43:06.750167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 20:43:06.778594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.778820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.779047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.779134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.779347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.779572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.787506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.788084 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.815169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.815370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.815637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.815880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.961862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:06.962058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.010010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.010234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.010781 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.011007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.074442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.074644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.074893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.075019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.075182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.075267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.075332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.075399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.075472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.075556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.075778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.075995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.100278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.100428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.100577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.100664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.100728 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.100795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.100949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.101088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.101227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.101308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.101308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.101480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.101594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.101719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.112738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.112866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.125423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.125660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.125822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.125907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.125972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.126046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.126109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.126240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.126314 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.126469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.126599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.126736 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.126811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.126901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 20:43:07.144139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.144374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 20:43:07.144579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.144706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.144865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 20:43:07.144950 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 20:43:07.145107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.145247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.145384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 20:43:07.145467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.145594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.145710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.145827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.145905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.146020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.146263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.156788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.157017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.157143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.157283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.157364 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.157529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.157655 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.157790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.165771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.165981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.169209 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.169415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.169542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.169679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.169761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.169948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.170075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.170213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.170298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.170483 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.170623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.170760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.170844 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.170964 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.171044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.171244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.171385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.171522 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.171604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.171725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.171805 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.171986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.198046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.198595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.199641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.199922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.200211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.200491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.205539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.205817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.206056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.206184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.206341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.206704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.213553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.213775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.214098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.214368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.218234 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.218520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.218805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.218937 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.219038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.219104 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.219338 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.219462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.219656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.219783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.219933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.220015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.220080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.220227 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.220383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.220668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.238566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.238842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.238965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.239258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.243428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.243860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.247098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.247390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.247622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.247906 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.252515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.252982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.256100 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.256559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.259790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.260042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.260210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.260364 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.260490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.260627 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.260760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.260963 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.261088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.261228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.261310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.261573 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.280073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.280379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.280504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.280641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.280743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.280897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.281023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.281160 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.288756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.288985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.290456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.290620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.290712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.290911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.294066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.294210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.294350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.294472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.294588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.294715 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.294828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.294955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.295103 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.295194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.308435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.308635 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.308733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.308933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.309060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.309217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.309358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.309439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.309515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.309754 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.311003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.311290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.311414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.311613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.329367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.329770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.332614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.333058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.335869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.336349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.337623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.338117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.339279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.339554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.339766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.339892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.340047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.340360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.347045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.347605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.348660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.349220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.351817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.352133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.352432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.352558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.352718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.352844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.353105 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.353398 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.361752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.362035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.362312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.362569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.362853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.363132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.370305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.370577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.370849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.370974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 20:43:07.371095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.371222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.371386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.371511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.371752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.371878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.371970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.372048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.372312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.372456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.372540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.372616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.372822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.373007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.373101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.373176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.373336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.373463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.373558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.373678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.373852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.373980 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.374215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.374489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 20:43:07.411365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564f2ddf49c0 pid: 80310 (default app) Jul 11 20:43:07.411983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564f2ddf49c0 pid: 80311 (default app) Jul 11 20:43:07.414776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2e7d1-91d3-43bd-bf79-e32a328d3c43 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:07.415071 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2e7d1-91d3-43bd-bf79-e32a328d3c43 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:07.415348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2e7d1-91d3-43bd-bf79-e32a328d3c43 None None] PATH_INFO: `/v3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:07.419594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Mon Jul 11 20:43:07 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:15.063899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd66243-ed6b-4073-99ec-3048090ee5d2 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:15.064518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd66243-ed6b-4073-99ec-3048090ee5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:15.064518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd66243-ed6b-4073-99ec-3048090ee5d2 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:15.068873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:43:15.075415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:15.076193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:15.076362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:15.173128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: INFO dbcounter [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] Registered counter for database keystone Jul 11 20:43:15.200124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG oslo_db.sqlalchemy.engines [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80310) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 20:43:15.207908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writer thread running {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 20:43:15.401151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG passlib.handlers.bcrypt [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80310) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 11 20:43:15.401581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG passlib.handlers.bcrypt [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80310) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 11 20:43:15.458300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:15.459873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:15.597115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 523 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 11 20:43:15.811822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd66243-ed6b-4073-99ec-3048090ee5d2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:15.886903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: INFO dbcounter [None req-2fd66243-ed6b-4073-99ec-3048090ee5d2 None None] Registered counter for database keystone Jul 11 20:43:15.908488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG oslo_db.sqlalchemy.engines [None req-2fd66243-ed6b-4073-99ec-3048090ee5d2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80311) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 20:43:15.918624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writer thread running {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 20:43:16.037359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd66243-ed6b-4073-99ec-3048090ee5d2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:16.042929 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65560f1c-19f7-4ded-8558-620badcd8726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:16.043591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65560f1c-19f7-4ded-8558-620badcd8726 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:16.043873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65560f1c-19f7-4ded-8558-620badcd8726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:16.044149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65560f1c-19f7-4ded-8558-620badcd8726 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:16.053296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65560f1c-19f7-4ded-8558-620badcd8726 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:16.104429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: WARNING keystone.common.rbac_enforcer.enforcer [None req-65560f1c-19f7-4ded-8558-620badcd8726 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 11 20:43:16.105004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65560f1c-19f7-4ded-8558-620badcd8726 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:16.115187 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Jul 11 20:43:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 20:43:16.125173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:16.144639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca2d0f-7588-4b92-8356-2f415609bf89 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:16.151204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef630ea-8342-4372-a0e2-fe8fc048ad9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:16.152388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef630ea-8342-4372-a0e2-fe8fc048ad9e admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:16.152679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef630ea-8342-4372-a0e2-fe8fc048ad9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:16.152952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef630ea-8342-4372-a0e2-fe8fc048ad9e admin admin] PATH_INFO: `/v3/projects/27db080494394cc5b886636221f8edb9` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:16.157328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef630ea-8342-4372-a0e2-fe8fc048ad9e admin admin] RBAC: Authorizing `identity:get_project(project_id=27db080494394cc5b886636221f8edb9)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:16.208762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fef630ea-8342-4372-a0e2-fe8fc048ad9e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 11 20:43:16.209096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef630ea-8342-4372-a0e2-fe8fc048ad9e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:16.213850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Jul 11 20:43:16 2022] GET /identity/v3/projects/27db080494394cc5b886636221f8edb9 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:18.199942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0b259-f386-4720-9d88-fa2871805c8b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:18.200549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0b259-f386-4720-9d88-fa2871805c8b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:18.200549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0b259-f386-4720-9d88-fa2871805c8b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:18.205281 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:18.211541 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831369ce-4b4e-4f30-90e8-5b863eacf9e4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:18.212091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831369ce-4b4e-4f30-90e8-5b863eacf9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:18.212091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831369ce-4b4e-4f30-90e8-5b863eacf9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:18.237558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-831369ce-4b4e-4f30-90e8-5b863eacf9e4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:18.239127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-831369ce-4b4e-4f30-90e8-5b863eacf9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:18.248452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 11 20:43:18.275369 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0b259-f386-4720-9d88-fa2871805c8b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:18.290406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0b259-f386-4720-9d88-fa2871805c8b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:18.295606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78407368-558b-40bf-ab51-5a642ba99c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:18.296291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407368-558b-40bf-ab51-5a642ba99c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:18.296575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407368-558b-40bf-ab51-5a642ba99c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:18.296833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407368-558b-40bf-ab51-5a642ba99c57 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:18.304917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78407368-558b-40bf-ab51-5a642ba99c57 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:18.306916 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78407368-558b-40bf-ab51-5a642ba99c57 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:18.315906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Jul 11 20:43:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 20:43:18.321809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831369ce-4b4e-4f30-90e8-5b863eacf9e4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:18.332256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831369ce-4b4e-4f30-90e8-5b863eacf9e4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:18.337195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb19ddc-fde9-4d55-93c6-a30d5ade6adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:18.338090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb19ddc-fde9-4d55-93c6-a30d5ade6adf admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:18.338376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb19ddc-fde9-4d55-93c6-a30d5ade6adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:18.338634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb19ddc-fde9-4d55-93c6-a30d5ade6adf admin admin] PATH_INFO: `/v3/users/5fc8fd4688454d8c8596f77d99812999` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:18.340082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb19ddc-fde9-4d55-93c6-a30d5ade6adf admin admin] RBAC: Authorizing `identity:get_user(user_id=5fc8fd4688454d8c8596f77d99812999)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:18.341956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb19ddc-fde9-4d55-93c6-a30d5ade6adf admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:18.343558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Mon Jul 11 20:43:18 2022] GET /identity/v3/users/5fc8fd4688454d8c8596f77d99812999 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:20.426713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17729dc7-174e-45c2-af0a-41785adf370c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:20.427294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17729dc7-174e-45c2-af0a-41785adf370c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:20.427294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17729dc7-174e-45c2-af0a-41785adf370c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:20.428664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:20.433882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc987d-f2a5-4c08-8e8b-f02e9d63fb84 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:20.434309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc987d-f2a5-4c08-8e8b-f02e9d63fb84 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:20.434538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc987d-f2a5-4c08-8e8b-f02e9d63fb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:20.451920 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a7dc987d-f2a5-4c08-8e8b-f02e9d63fb84 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:20.453500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a7dc987d-f2a5-4c08-8e8b-f02e9d63fb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:20.476962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b64bb4-4c10-42ea-970a-d2f45c9d2f16 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:20.477340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b64bb4-4c10-42ea-970a-d2f45c9d2f16 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:20.477688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b64bb4-4c10-42ea-970a-d2f45c9d2f16 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:20.479208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:20.484302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211e785-90e2-4bcb-9ca6-3076a9e384f5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:20.484586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211e785-90e2-4bcb-9ca6-3076a9e384f5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:20.484862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211e785-90e2-4bcb-9ca6-3076a9e384f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:20.523768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG passlib.handlers.bcrypt [None req-9211e785-90e2-4bcb-9ca6-3076a9e384f5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80311) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 11 20:43:20.524237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG passlib.handlers.bcrypt [None req-9211e785-90e2-4bcb-9ca6-3076a9e384f5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80311) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 11 20:43:20.537413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:20.559651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc987d-f2a5-4c08-8e8b-f02e9d63fb84 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:20.569374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9211e785-90e2-4bcb-9ca6-3076a9e384f5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:20.570953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9211e785-90e2-4bcb-9ca6-3076a9e384f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:20.584204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:20.667922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc987d-f2a5-4c08-8e8b-f02e9d63fb84 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:20.677993 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:43:20.701353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a235b-065c-46bf-bf3d-5a8ad45212c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:20.701975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211e785-90e2-4bcb-9ca6-3076a9e384f5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:20.702123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a235b-065c-46bf-bf3d-5a8ad45212c9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:20.702285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a235b-065c-46bf-bf3d-5a8ad45212c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:20.702560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a235b-065c-46bf-bf3d-5a8ad45212c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:20.768301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a235b-065c-46bf-bf3d-5a8ad45212c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:20.769817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a235b-065c-46bf-bf3d-5a8ad45212c9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:20.792485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:20.803562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211e785-90e2-4bcb-9ca6-3076a9e384f5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:20.805889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:20.843985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce9275-9e82-4089-93e4-1a51fbc26c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:20.845095 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce9275-9e82-4089-93e4-1a51fbc26c07 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:20.845384 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce9275-9e82-4089-93e4-1a51fbc26c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:20.845665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce9275-9e82-4089-93e4-1a51fbc26c07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:20.857798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce9275-9e82-4089-93e4-1a51fbc26c07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:20.868323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 7/14] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d7ce9275-9e82-4089-93e4-1a51fbc26c07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:20.869332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:20.870460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:20.871486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:43:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:43:20.874081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a235b-065c-46bf-bf3d-5a8ad45212c9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:20.878622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce9275-9e82-4089-93e4-1a51fbc26c07 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:20.975022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a235b-065c-46bf-bf3d-5a8ad45212c9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:20.979981 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce9275-9e82-4089-93e4-1a51fbc26c07 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:20.996922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:21.008958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eed9d5-7392-46dc-8656-f6d0dea23298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:21.009890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eed9d5-7392-46dc-8656-f6d0dea23298 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:21.010174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eed9d5-7392-46dc-8656-f6d0dea23298 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:21.010453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eed9d5-7392-46dc-8656-f6d0dea23298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:21.012431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253256b-fc81-43d0-8790-8813d064843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:21.013096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253256b-fc81-43d0-8790-8813d064843c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:21.013377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253256b-fc81-43d0-8790-8813d064843c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:21.013648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253256b-fc81-43d0-8790-8813d064843c None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:21.014403 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253256b-fc81-43d0-8790-8813d064843c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:21.014706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253256b-fc81-43d0-8790-8813d064843c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:21.016188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253256b-fc81-43d0-8790-8813d064843c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:21.020831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eed9d5-7392-46dc-8656-f6d0dea23298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:21.022310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eed9d5-7392-46dc-8656-f6d0dea23298 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:21.028001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 9/16] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:21.033634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253256b-fc81-43d0-8790-8813d064843c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d8eed9d5-7392-46dc-8656-f6d0dea23298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:43:21.043871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:21.044805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:21.045576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:43:21.045576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:43:21.045576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:43:21.045576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:21.045576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 8/17] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:21.051137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eed9d5-7392-46dc-8656-f6d0dea23298 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:21.062554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:21.134884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253256b-fc81-43d0-8790-8813d064843c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:21.154078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eed9d5-7392-46dc-8656-f6d0dea23298 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:21.171216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500ae03-d61e-469b-9351-d86db2fbed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:21.172126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500ae03-d61e-469b-9351-d86db2fbed65 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:21.172409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500ae03-d61e-469b-9351-d86db2fbed65 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:21.172669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500ae03-d61e-469b-9351-d86db2fbed65 None admin] PATH_INFO: `/v3/users/5fc8fd4688454d8c8596f77d99812999` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:21.174270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500ae03-d61e-469b-9351-d86db2fbed65 None admin] RBAC: Authorizing `identity:get_user(user_id=5fc8fd4688454d8c8596f77d99812999)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:21.175759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500ae03-d61e-469b-9351-d86db2fbed65 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:21.177408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:21 2022] GET /identity/v3/users/5fc8fd4688454d8c8596f77d99812999 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:21.183080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500ae03-d61e-469b-9351-d86db2fbed65 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:21.189564 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309e753-21c1-475c-a438-e83f9acffd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:21.190235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309e753-21c1-475c-a438-e83f9acffd21 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:21.190517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309e753-21c1-475c-a438-e83f9acffd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:21.190787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309e753-21c1-475c-a438-e83f9acffd21 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:21.191506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309e753-21c1-475c-a438-e83f9acffd21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:21.191806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309e753-21c1-475c-a438-e83f9acffd21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:21.193248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309e753-21c1-475c-a438-e83f9acffd21 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:21.210968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:21.285193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:21.293000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500ae03-d61e-469b-9351-d86db2fbed65 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:21.330631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582683d4-6013-4560-acd2-2ed8d2a74217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:21.331459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582683d4-6013-4560-acd2-2ed8d2a74217 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:21.331738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582683d4-6013-4560-acd2-2ed8d2a74217 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:21.332009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582683d4-6013-4560-acd2-2ed8d2a74217 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:21.333291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582683d4-6013-4560-acd2-2ed8d2a74217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:21.334785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582683d4-6013-4560-acd2-2ed8d2a74217 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:21.336741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:21.343279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309e753-21c1-475c-a438-e83f9acffd21 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:21.383031 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:21.450850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309e753-21c1-475c-a438-e83f9acffd21 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:21.486592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0ca72-83fb-4ea4-861f-3e671fe4ccd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:21.487261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0ca72-83fb-4ea4-861f-3e671fe4ccd0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:21.487538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0ca72-83fb-4ea4-861f-3e671fe4ccd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:21.487805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0ca72-83fb-4ea4-861f-3e671fe4ccd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:21.488710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0ca72-83fb-4ea4-861f-3e671fe4ccd0 None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.domain.id=default, user.id=5fc8fd4688454d8c8596f77d99812999) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:21.489005 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0ca72-83fb-4ea4-861f-3e671fe4ccd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:21.490456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0ca72-83fb-4ea4-861f-3e671fe4ccd0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:21.498108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Jul 11 20:43:21 2022] GET /identity/v3/role_assignments?user.id=5fc8fd4688454d8c8596f77d99812999&scope.domain.id=default&role.id=60bd1c44ce154338a7face8949d07281 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:23.630207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0fe99-00d1-4e02-88f0-f87c9f1df506 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:23.630747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0fe99-00d1-4e02-88f0-f87c9f1df506 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:23.630747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0fe99-00d1-4e02-88f0-f87c9f1df506 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:23.632157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:23.637711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f87e0-ed6b-45f2-b869-70232983df0b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:23.638117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f87e0-ed6b-45f2-b869-70232983df0b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:23.638284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f87e0-ed6b-45f2-b869-70232983df0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:23.665112 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-003f87e0-ed6b-45f2-b869-70232983df0b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:23.666368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-003f87e0-ed6b-45f2-b869-70232983df0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:23.729091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc4cfa-ba17-478a-8a96-acd7f42824df None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:23.729392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc4cfa-ba17-478a-8a96-acd7f42824df None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:23.729662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc4cfa-ba17-478a-8a96-acd7f42824df None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:23.731175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 13/23] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:43:23.738575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d47503-e6f1-4125-9436-60cac8c29f43 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:23.738858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d47503-e6f1-4125-9436-60cac8c29f43 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:23.739132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d47503-e6f1-4125-9436-60cac8c29f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:23.760139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-94d47503-e6f1-4125-9436-60cac8c29f43 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:23.761380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-94d47503-e6f1-4125-9436-60cac8c29f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:23.770484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 11/24] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:23.814446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f87e0-ed6b-45f2-b869-70232983df0b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:23.880558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:23.905479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 14/25] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:43:23.906607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664eece0-7f5a-4326-ab53-1d235c07ded0 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:23.906895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664eece0-7f5a-4326-ab53-1d235c07ded0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:23.907173 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664eece0-7f5a-4326-ab53-1d235c07ded0 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:23.915016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 15/26] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:23.950046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce795b-bce6-423a-9d17-568e66f0b878 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:23.950359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce795b-bce6-423a-9d17-568e66f0b878 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:23.950605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce795b-bce6-423a-9d17-568e66f0b878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:23.977580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-06ce795b-bce6-423a-9d17-568e66f0b878 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:23.978885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-06ce795b-bce6-423a-9d17-568e66f0b878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:24.020056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f87e0-ed6b-45f2-b869-70232983df0b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:24.056983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce59e9-ceed-49f2-a7e3-3972e4ee8fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:24.058272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce59e9-ceed-49f2-a7e3-3972e4ee8fad None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:24.058688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce59e9-ceed-49f2-a7e3-3972e4ee8fad None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:24.059095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce59e9-ceed-49f2-a7e3-3972e4ee8fad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:24.071358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce59e9-ceed-49f2-a7e3-3972e4ee8fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:24.073317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce59e9-ceed-49f2-a7e3-3972e4ee8fad None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-23ce59e9-ceed-49f2-a7e3-3972e4ee8fad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:43:24.096332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:24.097237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:24.098026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:43:24.098026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:43:24.098026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:43:24.098026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:24.100281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 12/27] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:43:24.101263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce59e9-ceed-49f2-a7e3-3972e4ee8fad None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:24.128551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 16/28] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:24.130235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c237a-5170-4195-9309-6c1fb7a04388 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:24.130876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c237a-5170-4195-9309-6c1fb7a04388 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:24.131324 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c237a-5170-4195-9309-6c1fb7a04388 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:24.137385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 17/29] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:24.138485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c237a-5170-4195-9309-6c1fb7a04388 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:24.168675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:24.227025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:24.254316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce59e9-ceed-49f2-a7e3-3972e4ee8fad None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:24.275257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c237a-5170-4195-9309-6c1fb7a04388 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:24.316130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3641265-1c21-401d-bf5d-df36022bbf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:24.316976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3641265-1c21-401d-bf5d-df36022bbf43 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:24.317310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3641265-1c21-401d-bf5d-df36022bbf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:24.317768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3641265-1c21-401d-bf5d-df36022bbf43 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:24.318703 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3641265-1c21-401d-bf5d-df36022bbf43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:24.319156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3641265-1c21-401d-bf5d-df36022bbf43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:24.320856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3641265-1c21-401d-bf5d-df36022bbf43 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:24.332138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b63853-4850-4df5-9554-09c1f63670e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:24.332138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b63853-4850-4df5-9554-09c1f63670e5 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:24.332138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b63853-4850-4df5-9554-09c1f63670e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:24.332138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b63853-4850-4df5-9554-09c1f63670e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:24.332470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b63853-4850-4df5-9554-09c1f63670e5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:24.333614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b63853-4850-4df5-9554-09c1f63670e5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:24.348249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 18/30] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:24.348249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45940b1-db96-4f89-8d57-9f2cf0695c5a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:24.348249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45940b1-db96-4f89-8d57-9f2cf0695c5a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:24.348249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45940b1-db96-4f89-8d57-9f2cf0695c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:24.364763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a45940b1-db96-4f89-8d57-9f2cf0695c5a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:24.366114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a45940b1-db96-4f89-8d57-9f2cf0695c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:24.451710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 19/31] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:24.460125 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45940b1-db96-4f89-8d57-9f2cf0695c5a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:24.480129 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:24.594961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45940b1-db96-4f89-8d57-9f2cf0695c5a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:24.610261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 13/32] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Jul 11 20:43:24 2022] POST /identity/v3/roles => generated 219 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:24.615703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b63853-4850-4df5-9554-09c1f63670e5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:24.635613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe4754-c24c-4265-b288-7dfa55e08ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:24.636574 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe4754-c24c-4265-b288-7dfa55e08ce3 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:24.636923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe4754-c24c-4265-b288-7dfa55e08ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:24.637304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe4754-c24c-4265-b288-7dfa55e08ce3 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:24.638152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe4754-c24c-4265-b288-7dfa55e08ce3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:24.639740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe4754-c24c-4265-b288-7dfa55e08ce3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:24.897169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 20/33] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Jul 11 20:43:24 2022] POST /identity/v3/roles => generated 225 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:24.898685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe4754-c24c-4265-b288-7dfa55e08ce3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:24.900956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b63853-4850-4df5-9554-09c1f63670e5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:24.983311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fa197-7050-482f-b722-c8ec9fcf74c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:24.984028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fa197-7050-482f-b722-c8ec9fcf74c6 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:24.984398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fa197-7050-482f-b722-c8ec9fcf74c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:24.984619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fa197-7050-482f-b722-c8ec9fcf74c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:24.985363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fa197-7050-482f-b722-c8ec9fcf74c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:24.987126 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fa197-7050-482f-b722-c8ec9fcf74c6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:25.001664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:25.004595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:25.033354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe4754-c24c-4265-b288-7dfa55e08ce3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:25.045655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 14/34] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:43:24 2022] POST /identity/v3/projects => generated 301 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:25.045655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c265cd-e980-4acb-9a79-3ff4484540c2 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.045655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c265cd-e980-4acb-9a79-3ff4484540c2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.045655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c265cd-e980-4acb-9a79-3ff4484540c2 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.046039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 15/35] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:25.053167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e347-0231-4e1b-b0f0-b7823ab86111 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.053553 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e347-0231-4e1b-b0f0-b7823ab86111 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.053836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e347-0231-4e1b-b0f0-b7823ab86111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.076983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-af23e347-0231-4e1b-b0f0-b7823ab86111 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:25.078736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-af23e347-0231-4e1b-b0f0-b7823ab86111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:25.105278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568d88e-3aef-48b2-af28-2f9af694b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:25.106468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568d88e-3aef-48b2-af28-2f9af694b93d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.106947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568d88e-3aef-48b2-af28-2f9af694b93d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.107438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568d88e-3aef-48b2-af28-2f9af694b93d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.129202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568d88e-3aef-48b2-af28-2f9af694b93d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9568d88e-3aef-48b2-af28-2f9af694b93d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:25.138983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:25.139884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:25.140902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 21/36] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:43:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:25.147757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568d88e-3aef-48b2-af28-2f9af694b93d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:25.163631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 16/37] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:25.195452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23e347-0231-4e1b-b0f0-b7823ab86111 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:25.279773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568d88e-3aef-48b2-af28-2f9af694b93d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:25.311105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23e347-0231-4e1b-b0f0-b7823ab86111 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:25.314874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:25.321487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11573bf-532d-4cff-91bb-bab1544d1e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:25.322182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11573bf-532d-4cff-91bb-bab1544d1e89 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.322828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11573bf-532d-4cff-91bb-bab1544d1e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.323116 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11573bf-532d-4cff-91bb-bab1544d1e89 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.324226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11573bf-532d-4cff-91bb-bab1544d1e89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:25.324547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11573bf-532d-4cff-91bb-bab1544d1e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:25.326150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11573bf-532d-4cff-91bb-bab1544d1e89 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:25.335678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 22/38] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:25.340131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a812dcf-0637-4c67-99ca-1e56c1d74a78 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.340131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a812dcf-0637-4c67-99ca-1e56c1d74a78 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.340131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a812dcf-0637-4c67-99ca-1e56c1d74a78 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.340131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 23/39] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:25.342774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a812dcf-0637-4c67-99ca-1e56c1d74a78 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:25.344734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:25.357507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b654b19-3646-4e35-b4c3-37675808bc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:25.358134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b654b19-3646-4e35-b4c3-37675808bc0d None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.358425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b654b19-3646-4e35-b4c3-37675808bc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.358703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b654b19-3646-4e35-b4c3-37675808bc0d None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.359504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b654b19-3646-4e35-b4c3-37675808bc0d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:25.361003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b654b19-3646-4e35-b4c3-37675808bc0d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:25.374119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 17/40] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Jul 11 20:43:25 2022] POST /identity/v3/roles => generated 223 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:25.375511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffc359-e06f-4ec6-9333-aec49cd9d05c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.375906 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffc359-e06f-4ec6-9333-aec49cd9d05c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.376069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffc359-e06f-4ec6-9333-aec49cd9d05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.406294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c4ffc359-e06f-4ec6-9333-aec49cd9d05c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:25.407538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c4ffc359-e06f-4ec6-9333-aec49cd9d05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:25.462323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a812dcf-0637-4c67-99ca-1e56c1d74a78 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:25.503832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 18/41] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:25.504026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a4959-2d4f-488a-9f68-b0a546ac323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:25.505027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a4959-2d4f-488a-9f68-b0a546ac323b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.505378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a4959-2d4f-488a-9f68-b0a546ac323b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.505726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a4959-2d4f-488a-9f68-b0a546ac323b None admin] PATH_INFO: `/v3/users/5fc8fd4688454d8c8596f77d99812999` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.507222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a4959-2d4f-488a-9f68-b0a546ac323b None admin] RBAC: Authorizing `identity:get_user(user_id=5fc8fd4688454d8c8596f77d99812999)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:25.508866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a4959-2d4f-488a-9f68-b0a546ac323b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:25.510900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 24/42] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:25 2022] GET /identity/v3/users/5fc8fd4688454d8c8596f77d99812999 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:25.516352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffc359-e06f-4ec6-9333-aec49cd9d05c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:25.529998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a4959-2d4f-488a-9f68-b0a546ac323b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:25.599387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:25.626709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffc359-e06f-4ec6-9333-aec49cd9d05c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:25.637837 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a4959-2d4f-488a-9f68-b0a546ac323b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:25.675039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec8e0d-7066-4fbe-aff9-d3f678843fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:25.675864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec8e0d-7066-4fbe-aff9-d3f678843fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.676168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec8e0d-7066-4fbe-aff9-d3f678843fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.676452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec8e0d-7066-4fbe-aff9-d3f678843fd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.677826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec8e0d-7066-4fbe-aff9-d3f678843fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:25.679342 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec8e0d-7066-4fbe-aff9-d3f678843fd3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:25.681129 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 25/43] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:25.687117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec8e0d-7066-4fbe-aff9-d3f678843fd3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:25.751654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:25.774386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d35dc-5d8d-465c-8347-7594db164391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:25.775616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d35dc-5d8d-465c-8347-7594db164391 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.775900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d35dc-5d8d-465c-8347-7594db164391 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.776210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d35dc-5d8d-465c-8347-7594db164391 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.777424 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d35dc-5d8d-465c-8347-7594db164391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:25.778890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d35dc-5d8d-465c-8347-7594db164391 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:25.780586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 19/44] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:25.786238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d35dc-5d8d-465c-8347-7594db164391 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:25.795980 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec8e0d-7066-4fbe-aff9-d3f678843fd3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:25.824880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:25.832218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597bf15-2701-40f2-8427-d4d8aff41164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:25.832898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597bf15-2701-40f2-8427-d4d8aff41164 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.833177 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597bf15-2701-40f2-8427-d4d8aff41164 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.833449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597bf15-2701-40f2-8427-d4d8aff41164 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.834165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597bf15-2701-40f2-8427-d4d8aff41164 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:25.835636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597bf15-2701-40f2-8427-d4d8aff41164 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:25.868663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 26/45] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:43:25 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:43:25.895493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d35dc-5d8d-465c-8347-7594db164391 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:25.931611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373be9c-221b-4b49-8ff4-d36a8a5ba92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:25.932766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373be9c-221b-4b49-8ff4-d36a8a5ba92e None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:25.933051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373be9c-221b-4b49-8ff4-d36a8a5ba92e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:25.933312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373be9c-221b-4b49-8ff4-d36a8a5ba92e None admin] PATH_INFO: `/v3/domains/default/users/5fc8fd4688454d8c8596f77d99812999/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:25.935480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373be9c-221b-4b49-8ff4-d36a8a5ba92e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5fc8fd4688454d8c8596f77d99812999, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:25.936990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373be9c-221b-4b49-8ff4-d36a8a5ba92e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:25.952934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 20/46] 10.222.0.20 () {58 vars in 1361 bytes} [Mon Jul 11 20:43:25 2022] PUT /identity/v3/domains/default/users/5fc8fd4688454d8c8596f77d99812999/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:43:28.084028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cb27f-474b-48da-b4ab-e86c9cfe3d9e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:28.084565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cb27f-474b-48da-b4ab-e86c9cfe3d9e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:28.084565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cb27f-474b-48da-b4ab-e86c9cfe3d9e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:28.085923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 27/47] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:28.091890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77840-8b9b-47f2-ac42-8d65bee4a6c6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:28.092341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77840-8b9b-47f2-ac42-8d65bee4a6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:28.092496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77840-8b9b-47f2-ac42-8d65bee4a6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:28.106133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1409d0-6110-456a-b6f0-259308387e20 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:28.106397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1409d0-6110-456a-b6f0-259308387e20 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:28.106706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1409d0-6110-456a-b6f0-259308387e20 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:28.108138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 28/48] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:28.110701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-56c77840-8b9b-47f2-ac42-8d65bee4a6c6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:28.111982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-56c77840-8b9b-47f2-ac42-8d65bee4a6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:28.115268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6837b9e-3181-4cec-8a6d-be85a0a16fa0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:28.115568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6837b9e-3181-4cec-8a6d-be85a0a16fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:28.116023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6837b9e-3181-4cec-8a6d-be85a0a16fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:28.138454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b6837b9e-3181-4cec-8a6d-be85a0a16fa0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:28.140319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b6837b9e-3181-4cec-8a6d-be85a0a16fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:28.157224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:28.203060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 21/49] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:28.227983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 29/50] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:28.228188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77840-8b9b-47f2-ac42-8d65bee4a6c6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:28.254744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6837b9e-3181-4cec-8a6d-be85a0a16fa0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:28.342599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77840-8b9b-47f2-ac42-8d65bee4a6c6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:28.366724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6837b9e-3181-4cec-8a6d-be85a0a16fa0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:28.377286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beea077-3a20-483e-9f7e-760dd40dddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:28.378132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beea077-3a20-483e-9f7e-760dd40dddc8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:28.378423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beea077-3a20-483e-9f7e-760dd40dddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:28.378708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beea077-3a20-483e-9f7e-760dd40dddc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:28.380430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:28.391477 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beea077-3a20-483e-9f7e-760dd40dddc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9beea077-3a20-483e-9f7e-760dd40dddc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:28.400502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:28.401337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:28.402154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 22/51] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:43:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:43:28.404298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827595ec-1209-4a29-a89f-f547c6e0fe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:28.405117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827595ec-1209-4a29-a89f-f547c6e0fe4d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:28.405404 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827595ec-1209-4a29-a89f-f547c6e0fe4d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:28.405684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827595ec-1209-4a29-a89f-f547c6e0fe4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:28.406905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827595ec-1209-4a29-a89f-f547c6e0fe4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:28.407679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beea077-3a20-483e-9f7e-760dd40dddc8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:28.408422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827595ec-1209-4a29-a89f-f547c6e0fe4d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:28.414546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 30/52] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:28.415649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827595ec-1209-4a29-a89f-f547c6e0fe4d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:28.439256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:28.523657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827595ec-1209-4a29-a89f-f547c6e0fe4d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:28.527962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beea077-3a20-483e-9f7e-760dd40dddc8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:28.560557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c5dd5-82cf-46c5-aa6c-1c09253fed0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:28.561251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c5dd5-82cf-46c5-aa6c-1c09253fed0a None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:28.561534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c5dd5-82cf-46c5-aa6c-1c09253fed0a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:28.561876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c5dd5-82cf-46c5-aa6c-1c09253fed0a None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:28.562520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c5dd5-82cf-46c5-aa6c-1c09253fed0a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:28.563988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c5dd5-82cf-46c5-aa6c-1c09253fed0a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:28.567730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be565b68-d3f6-46fd-bd0b-e2ba8f56c507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:28.568451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be565b68-d3f6-46fd-bd0b-e2ba8f56c507 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:28.568738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be565b68-d3f6-46fd-bd0b-e2ba8f56c507 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:28.569017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be565b68-d3f6-46fd-bd0b-e2ba8f56c507 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:28.569786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be565b68-d3f6-46fd-bd0b-e2ba8f56c507 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:28.570098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be565b68-d3f6-46fd-bd0b-e2ba8f56c507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:28.571552 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be565b68-d3f6-46fd-bd0b-e2ba8f56c507 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:28.585737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 23/53] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:28.591538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be565b68-d3f6-46fd-bd0b-e2ba8f56c507 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:28.591669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 31/54] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Jul 11 20:43:28 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:43:28.690050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:28.700311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be565b68-d3f6-46fd-bd0b-e2ba8f56c507 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:28.736232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339c3f0-ed86-40db-96e3-32bf1a14e58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:28.737139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339c3f0-ed86-40db-96e3-32bf1a14e58d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:28.737424 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339c3f0-ed86-40db-96e3-32bf1a14e58d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:28.737697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339c3f0-ed86-40db-96e3-32bf1a14e58d None admin] PATH_INFO: `/v3/users/5fc8fd4688454d8c8596f77d99812999` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:28.739044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339c3f0-ed86-40db-96e3-32bf1a14e58d None admin] RBAC: Authorizing `identity:get_user(user_id=5fc8fd4688454d8c8596f77d99812999)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:28.740578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339c3f0-ed86-40db-96e3-32bf1a14e58d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:28.742249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 24/55] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:28 2022] GET /identity/v3/users/5fc8fd4688454d8c8596f77d99812999 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:28.748300 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c5dd5-82cf-46c5-aa6c-1c09253fed0a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:28.783474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:28.973723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c5dd5-82cf-46c5-aa6c-1c09253fed0a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:29.008234 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd416ec-e4ff-4af3-befd-fc23413d9313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:29.009091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd416ec-e4ff-4af3-befd-fc23413d9313 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:29.009474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd416ec-e4ff-4af3-befd-fc23413d9313 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:29.009763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd416ec-e4ff-4af3-befd-fc23413d9313 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:29.011028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd416ec-e4ff-4af3-befd-fc23413d9313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:29.012533 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd416ec-e4ff-4af3-befd-fc23413d9313 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:29.014406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 32/56] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:29.020366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339c3f0-ed86-40db-96e3-32bf1a14e58d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:29.127856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339c3f0-ed86-40db-96e3-32bf1a14e58d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:29.164461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318a0b8-ec4f-45f3-a511-c6d10179a51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:29.165121 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318a0b8-ec4f-45f3-a511-c6d10179a51d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:29.165533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318a0b8-ec4f-45f3-a511-c6d10179a51d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:29.165980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318a0b8-ec4f-45f3-a511-c6d10179a51d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:29.166622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318a0b8-ec4f-45f3-a511-c6d10179a51d None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.domain.id=default, user.id=5fc8fd4688454d8c8596f77d99812999) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:29.166931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318a0b8-ec4f-45f3-a511-c6d10179a51d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:29.168433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318a0b8-ec4f-45f3-a511-c6d10179a51d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:29.175024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 25/57] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Jul 11 20:43:29 2022] GET /identity/v3/role_assignments?user.id=5fc8fd4688454d8c8596f77d99812999&scope.domain.id=default&role.id=60bd1c44ce154338a7face8949d07281 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:30.640853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2476226-7162-410b-a428-d0301d5e1ad1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:30.641383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2476226-7162-410b-a428-d0301d5e1ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:30.641467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2476226-7162-410b-a428-d0301d5e1ad1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:30.643580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 33/58] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:30.651575 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569003e-33fc-4147-85e3-6138e9979f7b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:30.652006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569003e-33fc-4147-85e3-6138e9979f7b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:30.652134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569003e-33fc-4147-85e3-6138e9979f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:30.669836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2569003e-33fc-4147-85e3-6138e9979f7b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:30.671036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2569003e-33fc-4147-85e3-6138e9979f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:30.689070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:30.742299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 26/59] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:43:30.766621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2476226-7162-410b-a428-d0301d5e1ad1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:30.879380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:30.884406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2476226-7162-410b-a428-d0301d5e1ad1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:30.925594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04b25e-95d6-43fd-b168-5634d5b8fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:30.926305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04b25e-95d6-43fd-b168-5634d5b8fbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:30.926584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04b25e-95d6-43fd-b168-5634d5b8fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:30.926861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04b25e-95d6-43fd-b168-5634d5b8fbc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:30.928211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04b25e-95d6-43fd-b168-5634d5b8fbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:30.929676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04b25e-95d6-43fd-b168-5634d5b8fbc9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:30.931344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 34/60] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:30.940650 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569003e-33fc-4147-85e3-6138e9979f7b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:31.046987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569003e-33fc-4147-85e3-6138e9979f7b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:31.082444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126ccfe-9278-48d3-8b03-fcdd5aecf4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:31.083110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ccfe-9278-48d3-8b03-fcdd5aecf4e2 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:31.083395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ccfe-9278-48d3-8b03-fcdd5aecf4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:31.083674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ccfe-9278-48d3-8b03-fcdd5aecf4e2 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:31.084414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126ccfe-9278-48d3-8b03-fcdd5aecf4e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:31.085888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126ccfe-9278-48d3-8b03-fcdd5aecf4e2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:31.113308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 27/61] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:43:30 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:34.061568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457913c5-9227-4c8e-9ae3-333ed382a63a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.062604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457913c5-9227-4c8e-9ae3-333ed382a63a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.063090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457913c5-9227-4c8e-9ae3-333ed382a63a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.064725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 35/62] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:34.070255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d64088-f688-4de0-a8b9-b3181a4f3301 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.071145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d64088-f688-4de0-a8b9-b3181a4f3301 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.071638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d64088-f688-4de0-a8b9-b3181a4f3301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.097159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-19d64088-f688-4de0-a8b9-b3181a4f3301 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:34.098647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-19d64088-f688-4de0-a8b9-b3181a4f3301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:34.101873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:34.178156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 28/63] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:34.202258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457913c5-9227-4c8e-9ae3-333ed382a63a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:34.321709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457913c5-9227-4c8e-9ae3-333ed382a63a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:34.368836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ebaa9-65b8-481d-95a8-19c77b8de4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:34.370014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ebaa9-65b8-481d-95a8-19c77b8de4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.370493 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ebaa9-65b8-481d-95a8-19c77b8de4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.370987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ebaa9-65b8-481d-95a8-19c77b8de4d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.385048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ebaa9-65b8-481d-95a8-19c77b8de4d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5e2ebaa9-65b8-481d-95a8-19c77b8de4d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:34.393184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:34.394019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:34.394755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:34.397119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 36/64] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:34.403290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d64088-f688-4de0-a8b9-b3181a4f3301 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:34.471597 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0195ea-a095-4ac4-b23d-8e3b984582d6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.472387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0195ea-a095-4ac4-b23d-8e3b984582d6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.472860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0195ea-a095-4ac4-b23d-8e3b984582d6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.474441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 37/65] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:34.479943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfb13e-56a2-40d2-a3c1-217e2225ed2c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.480529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfb13e-56a2-40d2-a3c1-217e2225ed2c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.481013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfb13e-56a2-40d2-a3c1-217e2225ed2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.499088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8bcfb13e-56a2-40d2-a3c1-217e2225ed2c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:34.500656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8bcfb13e-56a2-40d2-a3c1-217e2225ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:34.511344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d64088-f688-4de0-a8b9-b3181a4f3301 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:34.548219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f08054-1846-4646-9028-f8b86072d097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:34.549234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f08054-1846-4646-9028-f8b86072d097 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.549688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f08054-1846-4646-9028-f8b86072d097 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.550128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f08054-1846-4646-9028-f8b86072d097 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.551066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f08054-1846-4646-9028-f8b86072d097 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:34.551765 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:34.552373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f08054-1846-4646-9028-f8b86072d097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:34.553987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f08054-1846-4646-9028-f8b86072d097 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:34.564696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 29/66] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:34.570750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f08054-1846-4646-9028-f8b86072d097 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:34.576425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:34.646523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 38/67] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:43:34.689988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfb13e-56a2-40d2-a3c1-217e2225ed2c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:34.719105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f08054-1846-4646-9028-f8b86072d097 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:34.755381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf648ca8-747e-410f-bfe8-b69c8ccf7b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:34.756561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf648ca8-747e-410f-bfe8-b69c8ccf7b10 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.757091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf648ca8-747e-410f-bfe8-b69c8ccf7b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.757516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf648ca8-747e-410f-bfe8-b69c8ccf7b10 None admin] PATH_INFO: `/v3/users/0c0d3071ebef4f658f2da266190c1597` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.773297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf648ca8-747e-410f-bfe8-b69c8ccf7b10 None admin] RBAC: Authorizing `identity:get_user(user_id=0c0d3071ebef4f658f2da266190c1597)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:34.775137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf648ca8-747e-410f-bfe8-b69c8ccf7b10 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:34.777529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 30/68] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:34 2022] GET /identity/v3/users/0c0d3071ebef4f658f2da266190c1597 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:34.779064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a7d7-a142-4f9d-b009-c2be570624a0 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.779489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a7d7-a142-4f9d-b009-c2be570624a0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.779933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a7d7-a142-4f9d-b009-c2be570624a0 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.785768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 31/69] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:34.786657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a7d7-a142-4f9d-b009-c2be570624a0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:34.800692 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfb13e-56a2-40d2-a3c1-217e2225ed2c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:34.840246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53562405-52a5-460e-ba5e-2147088c8b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:34.841077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53562405-52a5-460e-ba5e-2147088c8b86 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.841366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53562405-52a5-460e-ba5e-2147088c8b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.841647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53562405-52a5-460e-ba5e-2147088c8b86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.851917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53562405-52a5-460e-ba5e-2147088c8b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:34.858135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-53562405-52a5-460e-ba5e-2147088c8b86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:34.859906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:34.860962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:34.861723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 39/70] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:43:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:43:34.862536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2fde-930a-42ab-b89b-0b3bc1ec204b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.862608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2fde-930a-42ab-b89b-0b3bc1ec204b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.862850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2fde-930a-42ab-b89b-0b3bc1ec204b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.881581 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1e4d2fde-930a-42ab-b89b-0b3bc1ec204b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:34.882759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1e4d2fde-930a-42ab-b89b-0b3bc1ec204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:34.896606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:34.908572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a7d7-a142-4f9d-b009-c2be570624a0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:34.945316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02b78d-62dc-455d-a4fd-5c188338ecf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:34.946049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02b78d-62dc-455d-a4fd-5c188338ecf6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.946365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02b78d-62dc-455d-a4fd-5c188338ecf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.946657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02b78d-62dc-455d-a4fd-5c188338ecf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.958787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 40/71] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:34.960650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1904d-40e1-4e11-bb18-ea4be6bba1af None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:34.961111 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1904d-40e1-4e11-bb18-ea4be6bba1af None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:34.961538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1904d-40e1-4e11-bb18-ea4be6bba1af None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:34.967507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 41/72] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:34.968525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1904d-40e1-4e11-bb18-ea4be6bba1af None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:35.025007 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02b78d-62dc-455d-a4fd-5c188338ecf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:35.026622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02b78d-62dc-455d-a4fd-5c188338ecf6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:35.083169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1904d-40e1-4e11-bb18-ea4be6bba1af None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:35.090099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:35.116762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b860cf-2a6f-401a-9b48-b2a040c29851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:35.117436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b860cf-2a6f-401a-9b48-b2a040c29851 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.117720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b860cf-2a6f-401a-9b48-b2a040c29851 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.118050 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b860cf-2a6f-401a-9b48-b2a040c29851 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.118804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b860cf-2a6f-401a-9b48-b2a040c29851 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:35.119111 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b860cf-2a6f-401a-9b48-b2a040c29851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:35.120553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b860cf-2a6f-401a-9b48-b2a040c29851 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:35.128685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 42/73] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:35.129764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383be9-fa34-4e65-bc78-bd1c3cd8917b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.130056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383be9-fa34-4e65-bc78-bd1c3cd8917b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.130337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383be9-fa34-4e65-bc78-bd1c3cd8917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.137027 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 32/74] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:35.137027 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02b78d-62dc-455d-a4fd-5c188338ecf6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:35.147300 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fa383be9-fa34-4e65-bc78-bd1c3cd8917b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:35.148481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fa383be9-fa34-4e65-bc78-bd1c3cd8917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:35.151602 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:35.215410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 43/75] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:35.216759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0ff18-7208-4b19-96c8-25711e3c5c4f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.217051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0ff18-7208-4b19-96c8-25711e3c5c4f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.217330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0ff18-7208-4b19-96c8-25711e3c5c4f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.218615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 44/76] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:35.219305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0ff18-7208-4b19-96c8-25711e3c5c4f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:35.235013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02b78d-62dc-455d-a4fd-5c188338ecf6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:35.269227 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4865a4a-7fa5-4ebd-ae74-d1f9b86d58c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:35.270042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4865a4a-7fa5-4ebd-ae74-d1f9b86d58c5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.270325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4865a4a-7fa5-4ebd-ae74-d1f9b86d58c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.270601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4865a4a-7fa5-4ebd-ae74-d1f9b86d58c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.280760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4865a4a-7fa5-4ebd-ae74-d1f9b86d58c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c4865a4a-7fa5-4ebd-ae74-d1f9b86d58c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:35.288783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:35.289716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:35.290490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:35.292332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 33/77] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:43:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:35.293109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4865a4a-7fa5-4ebd-ae74-d1f9b86d58c5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:35.317682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:35.323667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0ff18-7208-4b19-96c8-25711e3c5c4f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:35.364816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283dace-de28-43bd-b797-9dfaeb63238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:35.365716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283dace-de28-43bd-b797-9dfaeb63238f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.366017 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283dace-de28-43bd-b797-9dfaeb63238f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.366291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283dace-de28-43bd-b797-9dfaeb63238f None admin] PATH_INFO: `/v3/users/5fc8fd4688454d8c8596f77d99812999` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.367537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283dace-de28-43bd-b797-9dfaeb63238f None admin] RBAC: Authorizing `identity:get_user(user_id=5fc8fd4688454d8c8596f77d99812999)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:35.369194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283dace-de28-43bd-b797-9dfaeb63238f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:35.370864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 45/78] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:35 2022] GET /identity/v3/users/5fc8fd4688454d8c8596f77d99812999 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:35.372008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7f42d-75cd-48ee-8507-5f90bb7e04ca None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.372317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7f42d-75cd-48ee-8507-5f90bb7e04ca None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.372596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7f42d-75cd-48ee-8507-5f90bb7e04ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.389294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bdb7f42d-75cd-48ee-8507-5f90bb7e04ca None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:35.390485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bdb7f42d-75cd-48ee-8507-5f90bb7e04ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:35.392482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:35.411104 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4865a4a-7fa5-4ebd-ae74-d1f9b86d58c5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:35.448219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21e4ed-7a02-42a0-b39c-cab252b38e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:35.449100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21e4ed-7a02-42a0-b39c-cab252b38e1a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.449382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21e4ed-7a02-42a0-b39c-cab252b38e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.449654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21e4ed-7a02-42a0-b39c-cab252b38e1a None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.450894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21e4ed-7a02-42a0-b39c-cab252b38e1a None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:35.452779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21e4ed-7a02-42a0-b39c-cab252b38e1a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:35.454517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 34/79] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:35 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:35.455285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21e4ed-7a02-42a0-b39c-cab252b38e1a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:35.468630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 46/80] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:35.468630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7f42d-75cd-48ee-8507-5f90bb7e04ca None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:35.559502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:35.570830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7f42d-75cd-48ee-8507-5f90bb7e04ca None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:35.572370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21e4ed-7a02-42a0-b39c-cab252b38e1a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:35.604499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98be39-1beb-45c8-993f-84fcd27e5fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:35.605168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98be39-1beb-45c8-993f-84fcd27e5fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.605314 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293cae3-3acc-42f1-be7b-108e6ac5f14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:35.605449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98be39-1beb-45c8-993f-84fcd27e5fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.605722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98be39-1beb-45c8-993f-84fcd27e5fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.606116 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293cae3-3acc-42f1-be7b-108e6ac5f14a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.606396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293cae3-3acc-42f1-be7b-108e6ac5f14a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.606478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98be39-1beb-45c8-993f-84fcd27e5fb9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:35.606668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293cae3-3acc-42f1-be7b-108e6ac5f14a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.606778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98be39-1beb-45c8-993f-84fcd27e5fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:35.608192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98be39-1beb-45c8-993f-84fcd27e5fb9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:35.613147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:35.616540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 47/81] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:43:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:35.616985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293cae3-3acc-42f1-be7b-108e6ac5f14a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:35.617361 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98be39-1beb-45c8-993f-84fcd27e5fb9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-7293cae3-3acc-42f1-be7b-108e6ac5f14a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:35.625299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:35.626163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:35.626919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 35/82] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:35.627542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293cae3-3acc-42f1-be7b-108e6ac5f14a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:35.723757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98be39-1beb-45c8-993f-84fcd27e5fb9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:35.737162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293cae3-3acc-42f1-be7b-108e6ac5f14a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:35.757458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604896b-74bf-4e79-b4fb-e35c59347b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:35.758119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604896b-74bf-4e79-b4fb-e35c59347b28 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.758488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604896b-74bf-4e79-b4fb-e35c59347b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.759212 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604896b-74bf-4e79-b4fb-e35c59347b28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.771729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d91cd7-1d16-4096-ab73-4214e46e969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:35.772432 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d91cd7-1d16-4096-ab73-4214e46e969e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.772723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d91cd7-1d16-4096-ab73-4214e46e969e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.772992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d91cd7-1d16-4096-ab73-4214e46e969e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.773848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d91cd7-1d16-4096-ab73-4214e46e969e None admin] RBAC: Adding query filter params (role.id=1403cb7e4e264c68b63e467a2e433699, scope.project.id=3ad2772943334037bb33953f59d816e3, user.id=0c0d3071ebef4f658f2da266190c1597) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:35.774221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d91cd7-1d16-4096-ab73-4214e46e969e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:35.775701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d91cd7-1d16-4096-ab73-4214e46e969e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:35.781497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 36/83] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:35 2022] GET /identity/v3/role_assignments?user.id=0c0d3071ebef4f658f2da266190c1597&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:35.782295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d91cd7-1d16-4096-ab73-4214e46e969e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:35.798757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:35.835036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:35.837922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604896b-74bf-4e79-b4fb-e35c59347b28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:35.839807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604896b-74bf-4e79-b4fb-e35c59347b28 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:35.912758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d91cd7-1d16-4096-ab73-4214e46e969e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:35.949864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fa784-9885-4a46-bf06-f5647bcb3c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:35.950693 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fa784-9885-4a46-bf06-f5647bcb3c71 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:35.950997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fa784-9885-4a46-bf06-f5647bcb3c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:35.951270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fa784-9885-4a46-bf06-f5647bcb3c71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:35.954744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 48/84] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:35.955612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604896b-74bf-4e79-b4fb-e35c59347b28 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:35.956713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fa784-9885-4a46-bf06-f5647bcb3c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:35.958195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fa784-9885-4a46-bf06-f5647bcb3c71 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:35.964258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 37/85] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:35.964258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fa784-9885-4a46-bf06-f5647bcb3c71 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:36.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:36.063070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:36.066301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604896b-74bf-4e79-b4fb-e35c59347b28 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:36.073580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fa784-9885-4a46-bf06-f5647bcb3c71 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:36.102087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd213b-7ea7-4afd-bb38-b692c4389f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:36.103079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd213b-7ea7-4afd-bb38-b692c4389f68 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:36.103358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd213b-7ea7-4afd-bb38-b692c4389f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:36.103630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd213b-7ea7-4afd-bb38-b692c4389f68 None admin] PATH_INFO: `/v3/users/0c0d3071ebef4f658f2da266190c1597` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:36.104860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd213b-7ea7-4afd-bb38-b692c4389f68 None admin] RBAC: Authorizing `identity:get_user(user_id=0c0d3071ebef4f658f2da266190c1597)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:36.106313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd213b-7ea7-4afd-bb38-b692c4389f68 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:36.107957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 49/86] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:35 2022] GET /identity/v3/users/0c0d3071ebef4f658f2da266190c1597 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:36.112615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd213b-7ea7-4afd-bb38-b692c4389f68 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:36.114017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967883d-de43-4d60-8198-1356e57d88f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:36.114752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967883d-de43-4d60-8198-1356e57d88f4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:36.115048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967883d-de43-4d60-8198-1356e57d88f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:36.115333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967883d-de43-4d60-8198-1356e57d88f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:36.116126 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967883d-de43-4d60-8198-1356e57d88f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:36.116447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967883d-de43-4d60-8198-1356e57d88f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:36.117885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967883d-de43-4d60-8198-1356e57d88f4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:36.131066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 38/87] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:36.131066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967883d-de43-4d60-8198-1356e57d88f4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:36.218452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd213b-7ea7-4afd-bb38-b692c4389f68 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:36.234113 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967883d-de43-4d60-8198-1356e57d88f4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:36.257131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70030d51-57db-4470-a140-a2f1d940d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:36.258528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70030d51-57db-4470-a140-a2f1d940d23b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:36.259177 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70030d51-57db-4470-a140-a2f1d940d23b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:36.259390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70030d51-57db-4470-a140-a2f1d940d23b None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:36.261330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70030d51-57db-4470-a140-a2f1d940d23b None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:36.263982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70030d51-57db-4470-a140-a2f1d940d23b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:36.271030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 50/88] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:36 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:36.271030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70030d51-57db-4470-a140-a2f1d940d23b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:36.273796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5b95d-4e34-4594-a5de-d97767b32bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:36.274485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5b95d-4e34-4594-a5de-d97767b32bb9 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:36.274948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5b95d-4e34-4594-a5de-d97767b32bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:36.275229 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5b95d-4e34-4594-a5de-d97767b32bb9 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:36.275989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5b95d-4e34-4594-a5de-d97767b32bb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:36.277493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5b95d-4e34-4594-a5de-d97767b32bb9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:36.295909 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:36.302069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 39/89] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:43:36 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:36.302821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5b95d-4e34-4594-a5de-d97767b32bb9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:36.309565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:36.387089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70030d51-57db-4470-a140-a2f1d940d23b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:36.423824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f70a9e-8ab0-4b82-9911-9501a5c6c336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:36.424542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f70a9e-8ab0-4b82-9911-9501a5c6c336 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:36.425081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f70a9e-8ab0-4b82-9911-9501a5c6c336 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:36.425081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f70a9e-8ab0-4b82-9911-9501a5c6c336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:36.455298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5b95d-4e34-4594-a5de-d97767b32bb9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:36.500991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577d215-ec27-4212-be08-eea17660e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:36.501947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577d215-ec27-4212-be08-eea17660e1cd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:36.502236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577d215-ec27-4212-be08-eea17660e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:36.502568 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577d215-ec27-4212-be08-eea17660e1cd None admin] PATH_INFO: `/v3/users/0c0d3071ebef4f658f2da266190c1597` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:36.503923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577d215-ec27-4212-be08-eea17660e1cd None admin] RBAC: Authorizing `identity:get_user(user_id=0c0d3071ebef4f658f2da266190c1597)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:36.505496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577d215-ec27-4212-be08-eea17660e1cd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:36.506075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f70a9e-8ab0-4b82-9911-9501a5c6c336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:36.508090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 40/90] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:36 2022] GET /identity/v3/users/0c0d3071ebef4f658f2da266190c1597 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:36.508090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577d215-ec27-4212-be08-eea17660e1cd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:36.509128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f70a9e-8ab0-4b82-9911-9501a5c6c336 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:36.555061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:36.598192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:36.641520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 51/91] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:36.642384 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f70a9e-8ab0-4b82-9911-9501a5c6c336 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:36.643879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577d215-ec27-4212-be08-eea17660e1cd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:36.685786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57655fe5-b718-457c-9b73-e2e70c2d4e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:36.686457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57655fe5-b718-457c-9b73-e2e70c2d4e8e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:36.686737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57655fe5-b718-457c-9b73-e2e70c2d4e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:36.687003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57655fe5-b718-457c-9b73-e2e70c2d4e8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:36.687856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57655fe5-b718-457c-9b73-e2e70c2d4e8e None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=3ad2772943334037bb33953f59d816e3, user.id=5fc8fd4688454d8c8596f77d99812999) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:36.689027 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57655fe5-b718-457c-9b73-e2e70c2d4e8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:36.690519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57655fe5-b718-457c-9b73-e2e70c2d4e8e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:36.697155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 41/92] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:36 2022] GET /identity/v3/role_assignments?user.id=5fc8fd4688454d8c8596f77d99812999&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=60bd1c44ce154338a7face8949d07281 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:36.697992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57655fe5-b718-457c-9b73-e2e70c2d4e8e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:36.784331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f70a9e-8ab0-4b82-9911-9501a5c6c336 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:36.827876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d987a-0c4d-4089-baf5-e0fc2b317451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:36.828901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d987a-0c4d-4089-baf5-e0fc2b317451 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:36.829534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d987a-0c4d-4089-baf5-e0fc2b317451 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:36.830027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d987a-0c4d-4089-baf5-e0fc2b317451 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:36.834684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:36.874096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57655fe5-b718-457c-9b73-e2e70c2d4e8e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:36.914861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d987a-0c4d-4089-baf5-e0fc2b317451 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:36.916424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d987a-0c4d-4089-baf5-e0fc2b317451 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:36.939493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:36.953174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248249c5-ac0c-4f24-a07f-33cba4af532f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:36.954072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248249c5-ac0c-4f24-a07f-33cba4af532f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:36.954354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248249c5-ac0c-4f24-a07f-33cba4af532f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:36.954625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248249c5-ac0c-4f24-a07f-33cba4af532f None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:36.956134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248249c5-ac0c-4f24-a07f-33cba4af532f None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:36.957740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248249c5-ac0c-4f24-a07f-33cba4af532f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:36.959558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 42/93] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:36 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:36.965475 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248249c5-ac0c-4f24-a07f-33cba4af532f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:37.036343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 52/94] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:37.043188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d987a-0c4d-4089-baf5-e0fc2b317451 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:37.083940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:37.159507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d987a-0c4d-4089-baf5-e0fc2b317451 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:37.195377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ea9f0-6ea6-4e44-89f1-a65eab1a0454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:37.196418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ea9f0-6ea6-4e44-89f1-a65eab1a0454 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:37.196728 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ea9f0-6ea6-4e44-89f1-a65eab1a0454 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:37.196964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ea9f0-6ea6-4e44-89f1-a65eab1a0454 None admin] PATH_INFO: `/v3/projects/eddb8e2d61d24035bd48e7c45500ea0f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:37.198255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ea9f0-6ea6-4e44-89f1-a65eab1a0454 None admin] RBAC: Authorizing `identity:get_project(project_id=eddb8e2d61d24035bd48e7c45500ea0f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:37.199726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ea9f0-6ea6-4e44-89f1-a65eab1a0454 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:37.201684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 53/95] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:37 2022] GET /identity/v3/projects/eddb8e2d61d24035bd48e7c45500ea0f => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:37.207926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ea9f0-6ea6-4e44-89f1-a65eab1a0454 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:37.215294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248249c5-ac0c-4f24-a07f-33cba4af532f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:37.311172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ea9f0-6ea6-4e44-89f1-a65eab1a0454 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:37.312826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98138b8a-bfa1-40db-a1af-7f9e45b025b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:37.313515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98138b8a-bfa1-40db-a1af-7f9e45b025b3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:37.313802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98138b8a-bfa1-40db-a1af-7f9e45b025b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:37.314078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98138b8a-bfa1-40db-a1af-7f9e45b025b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:37.314382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:37.314947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98138b8a-bfa1-40db-a1af-7f9e45b025b3 None admin] RBAC: Adding query filter params (role.id=e2c9006dc77f473f8e1064f285400a98, scope.project.id=3ad2772943334037bb33953f59d816e3, user.id=0c0d3071ebef4f658f2da266190c1597) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:37.315265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98138b8a-bfa1-40db-a1af-7f9e45b025b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:37.316788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98138b8a-bfa1-40db-a1af-7f9e45b025b3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:37.331198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 43/96] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:36 2022] GET /identity/v3/role_assignments?user.id=0c0d3071ebef4f658f2da266190c1597&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=e2c9006dc77f473f8e1064f285400a98 => generated 259 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:37.350089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e89e85-abf4-40d8-af41-f4b877a9084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:37.350691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e89e85-abf4-40d8-af41-f4b877a9084b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:37.350977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e89e85-abf4-40d8-af41-f4b877a9084b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:37.351256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e89e85-abf4-40d8-af41-f4b877a9084b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:37.352173 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e89e85-abf4-40d8-af41-f4b877a9084b None admin] RBAC: Adding query filter params (role.id=1403cb7e4e264c68b63e467a2e433699, scope.project.id=eddb8e2d61d24035bd48e7c45500ea0f, user.id=0c0d3071ebef4f658f2da266190c1597) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:37.352502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e89e85-abf4-40d8-af41-f4b877a9084b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:37.354180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e89e85-abf4-40d8-af41-f4b877a9084b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:37.368302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 54/97] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:37 2022] GET /identity/v3/role_assignments?user.id=0c0d3071ebef4f658f2da266190c1597&scope.project.id=eddb8e2d61d24035bd48e7c45500ea0f&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:38.393466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428b86a-e194-4cc3-8a44-1d6b48c55e38 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:38.393980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428b86a-e194-4cc3-8a44-1d6b48c55e38 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:38.394051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428b86a-e194-4cc3-8a44-1d6b48c55e38 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:38.395424 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 44/98] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:38.401315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac8a4d-3098-40ac-bed9-aa9b3366d229 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:38.401717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac8a4d-3098-40ac-bed9-aa9b3366d229 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:38.401877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac8a4d-3098-40ac-bed9-aa9b3366d229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:38.423160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3dac8a4d-3098-40ac-bed9-aa9b3366d229 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:38.424386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3dac8a4d-3098-40ac-bed9-aa9b3366d229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:38.516619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 55/99] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:38.553512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428b86a-e194-4cc3-8a44-1d6b48c55e38 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:38.624025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:38.668425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428b86a-e194-4cc3-8a44-1d6b48c55e38 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:38.714064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3af2e-0660-409e-9d0c-914ec7c437e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:38.714897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3af2e-0660-409e-9d0c-914ec7c437e5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:38.715185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3af2e-0660-409e-9d0c-914ec7c437e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:38.715464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3af2e-0660-409e-9d0c-914ec7c437e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:38.730676 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3af2e-0660-409e-9d0c-914ec7c437e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-1bb3af2e-0660-409e-9d0c-914ec7c437e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:38.741700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:38.742547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:38.743408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:38.743961 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 45/100] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:38.749976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac8a4d-3098-40ac-bed9-aa9b3366d229 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:38.867357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:38.871654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac8a4d-3098-40ac-bed9-aa9b3366d229 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:38.911202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d2889-52e8-4806-9bff-c1b62357bcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:38.911883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d2889-52e8-4806-9bff-c1b62357bcbe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:38.912192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d2889-52e8-4806-9bff-c1b62357bcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:38.912472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d2889-52e8-4806-9bff-c1b62357bcbe None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:38.913230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d2889-52e8-4806-9bff-c1b62357bcbe None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:38.913535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d2889-52e8-4806-9bff-c1b62357bcbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:38.915306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d2889-52e8-4806-9bff-c1b62357bcbe None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:38.927778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 56/101] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:38.933133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3af2e-0660-409e-9d0c-914ec7c437e5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:39.056173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3af2e-0660-409e-9d0c-914ec7c437e5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:39.082416 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:39.101674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cc7d6-756d-4a68-b759-a840ba6b4afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:39.103145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cc7d6-756d-4a68-b759-a840ba6b4afc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:39.103626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cc7d6-756d-4a68-b759-a840ba6b4afc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:39.104091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cc7d6-756d-4a68-b759-a840ba6b4afc None admin] PATH_INFO: `/v3/users/0c0d3071ebef4f658f2da266190c1597` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:39.105533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cc7d6-756d-4a68-b759-a840ba6b4afc None admin] RBAC: Authorizing `identity:get_user(user_id=0c0d3071ebef4f658f2da266190c1597)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:39.107218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cc7d6-756d-4a68-b759-a840ba6b4afc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:39.110122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 46/102] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:38 2022] GET /identity/v3/users/0c0d3071ebef4f658f2da266190c1597 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:39.114711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d2889-52e8-4806-9bff-c1b62357bcbe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:39.243043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d2889-52e8-4806-9bff-c1b62357bcbe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:39.252966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb84ae-d9d3-4baf-9cbe-fcee9eb852fe None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:39.253551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb84ae-d9d3-4baf-9cbe-fcee9eb852fe None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:39.254047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb84ae-d9d3-4baf-9cbe-fcee9eb852fe None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:39.255910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 47/103] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:39.260927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab51e8-e51b-4e58-9614-af7681d31510 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:39.261468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab51e8-e51b-4e58-9614-af7681d31510 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:39.262093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab51e8-e51b-4e58-9614-af7681d31510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:39.290471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-45ab51e8-e51b-4e58-9614-af7681d31510 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:39.292195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-45ab51e8-e51b-4e58-9614-af7681d31510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:39.308540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b79ee-99df-4433-b684-975bb933ad78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:39.309557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b79ee-99df-4433-b684-975bb933ad78 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:39.309857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b79ee-99df-4433-b684-975bb933ad78 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:39.310143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b79ee-99df-4433-b684-975bb933ad78 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:39.316810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b79ee-99df-4433-b684-975bb933ad78 None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:39.316810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b79ee-99df-4433-b684-975bb933ad78 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:39.316810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 57/104] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:39 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:39.325166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b79ee-99df-4433-b684-975bb933ad78 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:39.384596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:39.395062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 48/105] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:39.435093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab51e8-e51b-4e58-9614-af7681d31510 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:39.531027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b79ee-99df-4433-b684-975bb933ad78 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:39.603696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a1731-8010-46bd-9cec-07661892d0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:39.604732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab51e8-e51b-4e58-9614-af7681d31510 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:39.605143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a1731-8010-46bd-9cec-07661892d0f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:39.605518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a1731-8010-46bd-9cec-07661892d0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:39.605800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a1731-8010-46bd-9cec-07661892d0f5 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3/users/0c0d3071ebef4f658f2da266190c1597/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:39.607799 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a1731-8010-46bd-9cec-07661892d0f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad2772943334037bb33953f59d816e3, user_id=0c0d3071ebef4f658f2da266190c1597, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:39.609470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a1731-8010-46bd-9cec-07661892d0f5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:39.633816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:39.657343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 58/106] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:43:39 2022] PUT /identity/v3/projects/3ad2772943334037bb33953f59d816e3/users/0c0d3071ebef4f658f2da266190c1597/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:43:39.660167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8235043-c627-41f5-82a3-3035797b9614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:39.660167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8235043-c627-41f5-82a3-3035797b9614 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:39.660167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8235043-c627-41f5-82a3-3035797b9614 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:39.660167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8235043-c627-41f5-82a3-3035797b9614 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:39.682587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8235043-c627-41f5-82a3-3035797b9614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a8235043-c627-41f5-82a3-3035797b9614 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:39.690577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:39.691423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:43:39.692195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:39.696972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 49/107] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:43:39.703198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a1731-8010-46bd-9cec-07661892d0f5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:39.855242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:39.878318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a1731-8010-46bd-9cec-07661892d0f5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:39.933866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4630c18-30b1-4988-9de4-42ab84d39657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:39.934570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630c18-30b1-4988-9de4-42ab84d39657 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:39.934856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630c18-30b1-4988-9de4-42ab84d39657 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:39.935132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630c18-30b1-4988-9de4-42ab84d39657 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:39.935906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4630c18-30b1-4988-9de4-42ab84d39657 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:39.936243 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4630c18-30b1-4988-9de4-42ab84d39657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:39.937783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4630c18-30b1-4988-9de4-42ab84d39657 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:39.952925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 59/108] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:39.958906 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8235043-c627-41f5-82a3-3035797b9614 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:40.047426 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563732ed-ec72-4428-aba3-28ee9e3854e3 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.047730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563732ed-ec72-4428-aba3-28ee9e3854e3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.047994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563732ed-ec72-4428-aba3-28ee9e3854e3 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.049479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 60/109] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:43:40.057318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88de67-fbc8-4c98-8b2f-08898010cfd7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.057603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88de67-fbc8-4c98-8b2f-08898010cfd7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.057882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88de67-fbc8-4c98-8b2f-08898010cfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.078306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-aa88de67-fbc8-4c98-8b2f-08898010cfd7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:40.079874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa88de67-fbc8-4c98-8b2f-08898010cfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:40.123301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8235043-c627-41f5-82a3-3035797b9614 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:40.174833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c8b79-42c3-4221-9268-e6a86b3d9b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:40.176189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c8b79-42c3-4221-9268-e6a86b3d9b37 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.176683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c8b79-42c3-4221-9268-e6a86b3d9b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.177519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c8b79-42c3-4221-9268-e6a86b3d9b37 None admin] PATH_INFO: `/v3/users/dacf5941777f42f28adcb47d5e613b75` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.193565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c8b79-42c3-4221-9268-e6a86b3d9b37 None admin] RBAC: Authorizing `identity:get_user(user_id=dacf5941777f42f28adcb47d5e613b75)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:40.195343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c8b79-42c3-4221-9268-e6a86b3d9b37 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:40.196614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 61/110] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:43:40.197928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 50/111] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:39 2022] GET /identity/v3/users/dacf5941777f42f28adcb47d5e613b75 => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:40.198028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eb564-27cf-4c7b-ab14-b202aa41d817 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.203086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eb564-27cf-4c7b-ab14-b202aa41d817 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.203086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eb564-27cf-4c7b-ab14-b202aa41d817 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.203086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 62/112] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:40.204588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c8b79-42c3-4221-9268-e6a86b3d9b37 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:40.209547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c664d8-fe84-418e-9f08-fd366e46f5e9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.210062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c664d8-fe84-418e-9f08-fd366e46f5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.210632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c664d8-fe84-418e-9f08-fd366e46f5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.240518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-47c664d8-fe84-418e-9f08-fd366e46f5e9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:40.242073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-47c664d8-fe84-418e-9f08-fd366e46f5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:40.298053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:40.320829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 63/113] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:40.322291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c664d8-fe84-418e-9f08-fd366e46f5e9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:40.337825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:40.366367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c8b79-42c3-4221-9268-e6a86b3d9b37 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:40.404399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20c2aa-951c-4ea0-921e-2eef9f70fe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:40.405992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20c2aa-951c-4ea0-921e-2eef9f70fe80 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.406410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20c2aa-951c-4ea0-921e-2eef9f70fe80 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.406921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20c2aa-951c-4ea0-921e-2eef9f70fe80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.450520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c664d8-fe84-418e-9f08-fd366e46f5e9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:40.483584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20c2aa-951c-4ea0-921e-2eef9f70fe80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:40.485105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20c2aa-951c-4ea0-921e-2eef9f70fe80 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:40.498534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0d27c-697f-4305-8c47-1cb979cdf70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:40.499439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0d27c-697f-4305-8c47-1cb979cdf70e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.499725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0d27c-697f-4305-8c47-1cb979cdf70e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.500000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0d27c-697f-4305-8c47-1cb979cdf70e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.510744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0d27c-697f-4305-8c47-1cb979cdf70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7ee0d27c-697f-4305-8c47-1cb979cdf70e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:40.518389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:40.519220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:40.519954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:40.524656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 64/114] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:43:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:40.526012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a04a2-873f-414c-ba05-552d00c0b4e6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.526306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a04a2-873f-414c-ba05-552d00c0b4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.526555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a04a2-873f-414c-ba05-552d00c0b4e6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.527888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 65/115] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:40.533055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a04a2-873f-414c-ba05-552d00c0b4e6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:40.534735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:40.601224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 51/116] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:40.603402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d7526-2d47-4641-b244-d6ffa6a20fed None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.603766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d7526-2d47-4641-b244-d6ffa6a20fed None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.604134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d7526-2d47-4641-b244-d6ffa6a20fed None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.605624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 52/117] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:40.609900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:40.609981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d7526-2d47-4641-b244-d6ffa6a20fed None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:40.642553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a04a2-873f-414c-ba05-552d00c0b4e6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:40.677151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac8a20-27f1-414a-9331-710c6ada7d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:40.677887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac8a20-27f1-414a-9331-710c6ada7d6c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.678180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac8a20-27f1-414a-9331-710c6ada7d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.678464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac8a20-27f1-414a-9331-710c6ada7d6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.679649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac8a20-27f1-414a-9331-710c6ada7d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:40.681248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac8a20-27f1-414a-9331-710c6ada7d6c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:40.682915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 66/118] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:40.684260 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e53a2d-508c-4c57-9eff-61d89187ae9c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.684550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e53a2d-508c-4c57-9eff-61d89187ae9c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.684831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e53a2d-508c-4c57-9eff-61d89187ae9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.708545 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-86e53a2d-508c-4c57-9eff-61d89187ae9c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:40.709753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-86e53a2d-508c-4c57-9eff-61d89187ae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:40.723199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d7526-2d47-4641-b244-d6ffa6a20fed None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:40.762037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173067b7-cf51-4944-b913-943bdee8e340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:40.762733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173067b7-cf51-4944-b913-943bdee8e340 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.763016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173067b7-cf51-4944-b913-943bdee8e340 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.763293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173067b7-cf51-4944-b913-943bdee8e340 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.764089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173067b7-cf51-4944-b913-943bdee8e340 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:40.764428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173067b7-cf51-4944-b913-943bdee8e340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:40.765838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173067b7-cf51-4944-b913-943bdee8e340 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:40.770976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:40.775035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 53/119] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:40.775944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173067b7-cf51-4944-b913-943bdee8e340 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:40.782569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 67/120] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:40.794308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e2b54-fabc-4163-824a-0a2b6731d725 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.794688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e2b54-fabc-4163-824a-0a2b6731d725 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.795036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e2b54-fabc-4163-824a-0a2b6731d725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.813571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-325e2b54-fabc-4163-824a-0a2b6731d725 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:40.816298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-325e2b54-fabc-4163-824a-0a2b6731d725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:40.881825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:40.888404 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 68/121] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:40.889374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e2b54-fabc-4163-824a-0a2b6731d725 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:40.889630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173067b7-cf51-4944-b913-943bdee8e340 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:40.934818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4850-363d-47dd-8c10-228cc4ed7961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:40.935758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4850-363d-47dd-8c10-228cc4ed7961 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:40.936045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4850-363d-47dd-8c10-228cc4ed7961 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:40.936362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4850-363d-47dd-8c10-228cc4ed7961 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:40.937567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e4850-363d-47dd-8c10-228cc4ed7961 None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:40.939040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e4850-363d-47dd-8c10-228cc4ed7961 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:40.940807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 54/122] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:40 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:40.941641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4850-363d-47dd-8c10-228cc4ed7961 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:40.996888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e2b54-fabc-4163-824a-0a2b6731d725 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:41.010119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:41.032847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2995a-f4ea-43f7-bb07-ef68ebaa6263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:41.033921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2995a-f4ea-43f7-bb07-ef68ebaa6263 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:41.033921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2995a-f4ea-43f7-bb07-ef68ebaa6263 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:41.034228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2995a-f4ea-43f7-bb07-ef68ebaa6263 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:41.035063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2995a-f4ea-43f7-bb07-ef68ebaa6263 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:41.036679 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2995a-f4ea-43f7-bb07-ef68ebaa6263 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:41.046700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4850-363d-47dd-8c10-228cc4ed7961 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:41.066725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 69/123] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:43:40 2022] POST /identity/v3/projects => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:41.067524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2995a-f4ea-43f7-bb07-ef68ebaa6263 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:41.100599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd855d63-d662-4351-8af4-800639ba6f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:41.102751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd855d63-d662-4351-8af4-800639ba6f47 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:41.103412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd855d63-d662-4351-8af4-800639ba6f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:41.104004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd855d63-d662-4351-8af4-800639ba6f47 None admin] PATH_INFO: `/v3/users/5fc8fd4688454d8c8596f77d99812999` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:41.105881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd855d63-d662-4351-8af4-800639ba6f47 None admin] RBAC: Authorizing `identity:get_user(user_id=5fc8fd4688454d8c8596f77d99812999)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:41.108284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd855d63-d662-4351-8af4-800639ba6f47 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:41.115901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 55/124] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:40 2022] GET /identity/v3/users/5fc8fd4688454d8c8596f77d99812999 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:41.115901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd855d63-d662-4351-8af4-800639ba6f47 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:41.135406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:41.202682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2995a-f4ea-43f7-bb07-ef68ebaa6263 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:41.242749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4ecbe-cbfa-462a-accd-3ad9809ee075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:41.243574 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4ecbe-cbfa-462a-accd-3ad9809ee075 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:41.243865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4ecbe-cbfa-462a-accd-3ad9809ee075 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:41.244170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4ecbe-cbfa-462a-accd-3ad9809ee075 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:41.253421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd855d63-d662-4351-8af4-800639ba6f47 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:41.257071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4ecbe-cbfa-462a-accd-3ad9809ee075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-acb4ecbe-cbfa-462a-accd-3ad9809ee075 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:41.266737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:41.267580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 70/125] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:41.269051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4ecbe-cbfa-462a-accd-3ad9809ee075 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:41.302577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584b130-7003-4ca3-b3d7-85cade920573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:41.303426 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584b130-7003-4ca3-b3d7-85cade920573 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:41.303706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584b130-7003-4ca3-b3d7-85cade920573 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:41.303983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584b130-7003-4ca3-b3d7-85cade920573 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:41.307573 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:41.316767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584b130-7003-4ca3-b3d7-85cade920573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c584b130-7003-4ca3-b3d7-85cade920573 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:41.325133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:41.326021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:41.327146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 56/126] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:43:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:41.327758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584b130-7003-4ca3-b3d7-85cade920573 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:41.388033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4ecbe-cbfa-462a-accd-3ad9809ee075 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:41.399031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:41.426403 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec1bf9-4f3c-4a8a-ab69-a28ffd9a1f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:41.427128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec1bf9-4f3c-4a8a-ab69-a28ffd9a1f49 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:41.427424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec1bf9-4f3c-4a8a-ab69-a28ffd9a1f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:41.427708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec1bf9-4f3c-4a8a-ab69-a28ffd9a1f49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:41.428594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec1bf9-4f3c-4a8a-ab69-a28ffd9a1f49 None admin] RBAC: Adding query filter params (role.id=989e97787f6c43b5aa2603cc1b732f40, scope.project.id=3ad2772943334037bb33953f59d816e3, user.id=dacf5941777f42f28adcb47d5e613b75) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:41.428915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec1bf9-4f3c-4a8a-ab69-a28ffd9a1f49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:41.430688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec1bf9-4f3c-4a8a-ab69-a28ffd9a1f49 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:41.440822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 71/127] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:41 2022] GET /identity/v3/role_assignments?user.id=dacf5941777f42f28adcb47d5e613b75&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=989e97787f6c43b5aa2603cc1b732f40 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:41.441664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec1bf9-4f3c-4a8a-ab69-a28ffd9a1f49 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:41.486464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584b130-7003-4ca3-b3d7-85cade920573 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:41.569957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec1bf9-4f3c-4a8a-ab69-a28ffd9a1f49 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:41.592870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56090f-4d32-4039-a28f-dcea88ec75eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:41.593773 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56090f-4d32-4039-a28f-dcea88ec75eb None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:41.594056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56090f-4d32-4039-a28f-dcea88ec75eb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:41.594330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56090f-4d32-4039-a28f-dcea88ec75eb None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:41.595588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56090f-4d32-4039-a28f-dcea88ec75eb None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:41.597257 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56090f-4d32-4039-a28f-dcea88ec75eb None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:41.599350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 57/128] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:41 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:41.600167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56090f-4d32-4039-a28f-dcea88ec75eb None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:41.610214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a522147-ac09-493d-80c3-90aa9cbe0687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:41.611240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a522147-ac09-493d-80c3-90aa9cbe0687 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:41.611536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a522147-ac09-493d-80c3-90aa9cbe0687 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:41.611822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a522147-ac09-493d-80c3-90aa9cbe0687 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:41.612751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a522147-ac09-493d-80c3-90aa9cbe0687 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:41.613160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a522147-ac09-493d-80c3-90aa9cbe0687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:41.614709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a522147-ac09-493d-80c3-90aa9cbe0687 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:41.627467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 72/129] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:41.632831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a522147-ac09-493d-80c3-90aa9cbe0687 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:41.677744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:41.740452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a522147-ac09-493d-80c3-90aa9cbe0687 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:41.741868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:41.775151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a925db9-d5a8-4c9a-a588-5ed91812d887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:41.776088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a925db9-d5a8-4c9a-a588-5ed91812d887 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:41.776389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a925db9-d5a8-4c9a-a588-5ed91812d887 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:41.776671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a925db9-d5a8-4c9a-a588-5ed91812d887 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3/users/5fc8fd4688454d8c8596f77d99812999/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:41.778765 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a925db9-d5a8-4c9a-a588-5ed91812d887 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad2772943334037bb33953f59d816e3, user_id=5fc8fd4688454d8c8596f77d99812999, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:41.780303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a925db9-d5a8-4c9a-a588-5ed91812d887 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:41.804147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 73/130] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:43:41 2022] PUT /identity/v3/projects/3ad2772943334037bb33953f59d816e3/users/5fc8fd4688454d8c8596f77d99812999/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:43:41.804147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a925db9-d5a8-4c9a-a588-5ed91812d887 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:41.834028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56090f-4d32-4039-a28f-dcea88ec75eb None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:41.896832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0671eeb-1242-40ab-8a1b-e73adce841e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:41.897955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0671eeb-1242-40ab-8a1b-e73adce841e2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:41.898434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0671eeb-1242-40ab-8a1b-e73adce841e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:41.898980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0671eeb-1242-40ab-8a1b-e73adce841e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:41.900029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0671eeb-1242-40ab-8a1b-e73adce841e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:41.900570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0671eeb-1242-40ab-8a1b-e73adce841e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:41.902252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0671eeb-1242-40ab-8a1b-e73adce841e2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:41.920549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 58/131] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:43:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:41.924169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a925db9-d5a8-4c9a-a588-5ed91812d887 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:41.924474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:41.926831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0671eeb-1242-40ab-8a1b-e73adce841e2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:41.961528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51a475-ac61-4e5d-9763-b4b8c3a304a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:41.962625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51a475-ac61-4e5d-9763-b4b8c3a304a3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:41.963058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51a475-ac61-4e5d-9763-b4b8c3a304a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:41.963470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51a475-ac61-4e5d-9763-b4b8c3a304a3 None admin] PATH_INFO: `/v3/users/0c0d3071ebef4f658f2da266190c1597` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:41.965273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51a475-ac61-4e5d-9763-b4b8c3a304a3 None admin] RBAC: Authorizing `identity:get_user(user_id=0c0d3071ebef4f658f2da266190c1597)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:41.972163 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51a475-ac61-4e5d-9763-b4b8c3a304a3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:41.972383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 74/132] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:41 2022] GET /identity/v3/users/0c0d3071ebef4f658f2da266190c1597 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:41.975552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51a475-ac61-4e5d-9763-b4b8c3a304a3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:42.084299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51a475-ac61-4e5d-9763-b4b8c3a304a3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:42.123131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0671eeb-1242-40ab-8a1b-e73adce841e2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:42.152869 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f19d32-17ed-4a80-912d-ed568cb6ce1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:42.153869 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f19d32-17ed-4a80-912d-ed568cb6ce1e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:42.154197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f19d32-17ed-4a80-912d-ed568cb6ce1e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:42.154459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f19d32-17ed-4a80-912d-ed568cb6ce1e None admin] PATH_INFO: `/v3/projects/eddb8e2d61d24035bd48e7c45500ea0f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:42.155780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f19d32-17ed-4a80-912d-ed568cb6ce1e None admin] RBAC: Authorizing `identity:get_project(project_id=eddb8e2d61d24035bd48e7c45500ea0f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:42.158961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f19d32-17ed-4a80-912d-ed568cb6ce1e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:42.159071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 75/133] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:41 2022] GET /identity/v3/projects/eddb8e2d61d24035bd48e7c45500ea0f => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:42.166334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f19d32-17ed-4a80-912d-ed568cb6ce1e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:42.189848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:42.219921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1c3f-3f78-4e7a-8b19-60295dfee9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:42.221116 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1c3f-3f78-4e7a-8b19-60295dfee9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:42.221410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1c3f-3f78-4e7a-8b19-60295dfee9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:42.221689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1c3f-3f78-4e7a-8b19-60295dfee9b4 None admin] PATH_INFO: `/v3/users/0c0d3071ebef4f658f2da266190c1597` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:42.223518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d1c3f-3f78-4e7a-8b19-60295dfee9b4 None admin] RBAC: Authorizing `identity:get_user(user_id=0c0d3071ebef4f658f2da266190c1597)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:42.225327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d1c3f-3f78-4e7a-8b19-60295dfee9b4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:42.227750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 59/134] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:41 2022] GET /identity/v3/users/0c0d3071ebef4f658f2da266190c1597 => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:42.228317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:42.229700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f737823-8a47-4d35-9bf9-21b40ed2b97e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:42.229990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f737823-8a47-4d35-9bf9-21b40ed2b97e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:42.230265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f737823-8a47-4d35-9bf9-21b40ed2b97e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:42.231789 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 60/135] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:42.237317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f737823-8a47-4d35-9bf9-21b40ed2b97e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:42.350392 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f737823-8a47-4d35-9bf9-21b40ed2b97e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:42.391439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f19d32-17ed-4a80-912d-ed568cb6ce1e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:42.426479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fac55-d07f-489f-80be-596dfe4f4c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:42.427366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fac55-d07f-489f-80be-596dfe4f4c76 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:42.427653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fac55-d07f-489f-80be-596dfe4f4c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:42.427930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fac55-d07f-489f-80be-596dfe4f4c76 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:42.429842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fac55-d07f-489f-80be-596dfe4f4c76 None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:42.431336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fac55-d07f-489f-80be-596dfe4f4c76 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:42.433081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513c95d-8177-43c8-90a1-bb3333c97c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:42.433988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513c95d-8177-43c8-90a1-bb3333c97c88 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:42.434269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513c95d-8177-43c8-90a1-bb3333c97c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:42.434547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513c95d-8177-43c8-90a1-bb3333c97c88 None admin] PATH_INFO: `/v3/projects/eddb8e2d61d24035bd48e7c45500ea0f/users/0c0d3071ebef4f658f2da266190c1597/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:42.436557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 61/136] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:42 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:42.438283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de4d63-f60d-45f9-b654-ea8c63d3878b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:42.439160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513c95d-8177-43c8-90a1-bb3333c97c88 None admin] RBAC: Authorizing `identity:create_grant(project_id=eddb8e2d61d24035bd48e7c45500ea0f, user_id=0c0d3071ebef4f658f2da266190c1597, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:42.440700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513c95d-8177-43c8-90a1-bb3333c97c88 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:42.442000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de4d63-f60d-45f9-b654-ea8c63d3878b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:42.443346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de4d63-f60d-45f9-b654-ea8c63d3878b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:42.463537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 76/137] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:43:42 2022] PUT /identity/v3/projects/eddb8e2d61d24035bd48e7c45500ea0f/users/0c0d3071ebef4f658f2da266190c1597/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:43:42.465936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513c95d-8177-43c8-90a1-bb3333c97c88 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:42.484324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-78de4d63-f60d-45f9-b654-ea8c63d3878b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:42.486504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-78de4d63-f60d-45f9-b654-ea8c63d3878b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:42.595466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:42.597375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:42.614185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513c95d-8177-43c8-90a1-bb3333c97c88 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:42.625613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 62/138] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:42.675685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b565827-c64d-40eb-8a7c-711ac0b51f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:42.676676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b565827-c64d-40eb-8a7c-711ac0b51f25 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:42.676957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b565827-c64d-40eb-8a7c-711ac0b51f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:42.677292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b565827-c64d-40eb-8a7c-711ac0b51f25 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3/users/0c0d3071ebef4f658f2da266190c1597/roles/e2c9006dc77f473f8e1064f285400a98` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:42.682177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de4d63-f60d-45f9-b654-ea8c63d3878b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:42.682346 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b565827-c64d-40eb-8a7c-711ac0b51f25 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad2772943334037bb33953f59d816e3, user_id=0c0d3071ebef4f658f2da266190c1597, role_id=e2c9006dc77f473f8e1064f285400a98)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:42.683942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b565827-c64d-40eb-8a7c-711ac0b51f25 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:42.700176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 77/139] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:43:42 2022] PUT /identity/v3/projects/3ad2772943334037bb33953f59d816e3/users/0c0d3071ebef4f658f2da266190c1597/roles/e2c9006dc77f473f8e1064f285400a98 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:43:42.815126 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de4d63-f60d-45f9-b654-ea8c63d3878b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:42.857478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cbded-d26f-43c8-8409-19f8884102e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:42.858321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cbded-d26f-43c8-8409-19f8884102e7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:42.858603 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cbded-d26f-43c8-8409-19f8884102e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:42.858881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cbded-d26f-43c8-8409-19f8884102e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:42.873915 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8cbded-d26f-43c8-8409-19f8884102e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-cf8cbded-d26f-43c8-8409-19f8884102e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:42.885655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:42.886542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:42.887324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 63/140] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:43:42.894849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b565827-c64d-40eb-8a7c-711ac0b51f25 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:43.009840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b565827-c64d-40eb-8a7c-711ac0b51f25 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:43.047640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0eda5-af98-444e-ab99-9c4885a9fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:43.048715 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0eda5-af98-444e-ab99-9c4885a9fec7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:43.049208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0eda5-af98-444e-ab99-9c4885a9fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:43.049689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0eda5-af98-444e-ab99-9c4885a9fec7 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:43.050648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0eda5-af98-444e-ab99-9c4885a9fec7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:43.051153 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0eda5-af98-444e-ab99-9c4885a9fec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:43.052858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0eda5-af98-444e-ab99-9c4885a9fec7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:43.068299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 78/141] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:43.073876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cbded-d26f-43c8-8409-19f8884102e7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:43.172489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:43.236269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cbded-d26f-43c8-8409-19f8884102e7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:43.285727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5e6ac-976b-4d80-8c34-c30981dcec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:43.286635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5e6ac-976b-4d80-8c34-c30981dcec2d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:43.286922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5e6ac-976b-4d80-8c34-c30981dcec2d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:43.287202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5e6ac-976b-4d80-8c34-c30981dcec2d None admin] PATH_INFO: `/v3/users/0c0d3071ebef4f658f2da266190c1597` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:43.288461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5e6ac-976b-4d80-8c34-c30981dcec2d None admin] RBAC: Authorizing `identity:get_user(user_id=0c0d3071ebef4f658f2da266190c1597)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:43.290078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5e6ac-976b-4d80-8c34-c30981dcec2d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:43.291744 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 64/142] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:43 2022] GET /identity/v3/users/0c0d3071ebef4f658f2da266190c1597 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:43.297310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0eda5-af98-444e-ab99-9c4885a9fec7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:43.336207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:43.451395 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0eda5-af98-444e-ab99-9c4885a9fec7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:43.505722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b7ff7-6e0d-4ccf-adac-6d0154d6281c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:43.506764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b7ff7-6e0d-4ccf-adac-6d0154d6281c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:43.507252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b7ff7-6e0d-4ccf-adac-6d0154d6281c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:43.507730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b7ff7-6e0d-4ccf-adac-6d0154d6281c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:43.599557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b7ff7-6e0d-4ccf-adac-6d0154d6281c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:43.601908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b7ff7-6e0d-4ccf-adac-6d0154d6281c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:43.625071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:43.725054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 79/143] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:43.730789 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5e6ac-976b-4d80-8c34-c30981dcec2d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:43.867478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5e6ac-976b-4d80-8c34-c30981dcec2d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:43.896297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:43.914395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425205ba-4371-4d52-ab2e-9a4fe2a01ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:43.915296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425205ba-4371-4d52-ab2e-9a4fe2a01ece None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:43.915586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425205ba-4371-4d52-ab2e-9a4fe2a01ece None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:43.915874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425205ba-4371-4d52-ab2e-9a4fe2a01ece None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:43.917143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425205ba-4371-4d52-ab2e-9a4fe2a01ece None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:43.918638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425205ba-4371-4d52-ab2e-9a4fe2a01ece None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:43.920418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 65/144] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:43 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:43.926500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b7ff7-6e0d-4ccf-adac-6d0154d6281c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:43.980185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd2c64-244a-4a61-855a-b811eace2708 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:43.980488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd2c64-244a-4a61-855a-b811eace2708 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:43.980743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd2c64-244a-4a61-855a-b811eace2708 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:43.982145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 66/145] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:43.987025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c26e4-c600-4931-8685-e732bc0db54f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:43.987319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c26e4-c600-4931-8685-e732bc0db54f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:43.987604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c26e4-c600-4931-8685-e732bc0db54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:44.009628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-415c26e4-c600-4931-8685-e732bc0db54f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:44.010840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-415c26e4-c600-4931-8685-e732bc0db54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:44.080259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b7ff7-6e0d-4ccf-adac-6d0154d6281c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:44.095764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 67/146] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:43:44.109190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:44.135595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05522a-b011-4bb9-8398-9108b55444bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:44.136591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05522a-b011-4bb9-8398-9108b55444bb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:44.137038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05522a-b011-4bb9-8398-9108b55444bb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:44.137466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05522a-b011-4bb9-8398-9108b55444bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:44.139470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c26e4-c600-4931-8685-e732bc0db54f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:44.142732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05522a-b011-4bb9-8398-9108b55444bb None admin] RBAC: Adding query filter params (role.id=1403cb7e4e264c68b63e467a2e433699, scope.project.id=3ad2772943334037bb33953f59d816e3, user.id=0c0d3071ebef4f658f2da266190c1597) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:44.143222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05522a-b011-4bb9-8398-9108b55444bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:44.153657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05522a-b011-4bb9-8398-9108b55444bb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:44.160797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 80/147] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:43 2022] GET /identity/v3/role_assignments?user.id=0c0d3071ebef4f658f2da266190c1597&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:44.287255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c26e4-c600-4931-8685-e732bc0db54f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:44.315138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:44.340626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b540d-fa21-4346-8168-7490bd3f273d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:44.341474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b540d-fa21-4346-8168-7490bd3f273d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:44.341761 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b540d-fa21-4346-8168-7490bd3f273d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:44.342043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b540d-fa21-4346-8168-7490bd3f273d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:44.343370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b540d-fa21-4346-8168-7490bd3f273d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:44.345043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b540d-fa21-4346-8168-7490bd3f273d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:44.346744 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 68/148] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:44.352500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05522a-b011-4bb9-8398-9108b55444bb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:44.513605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05522a-b011-4bb9-8398-9108b55444bb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:44.567248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dce123-287c-402e-8061-6689779bfe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:44.567848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce123-287c-402e-8061-6689779bfe45 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:44.568135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce123-287c-402e-8061-6689779bfe45 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:44.568416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce123-287c-402e-8061-6689779bfe45 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:44.569144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dce123-287c-402e-8061-6689779bfe45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:44.570661 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dce123-287c-402e-8061-6689779bfe45 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:44.610392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 81/149] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:43:44 2022] POST /identity/v3/users => generated 283 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:44.779372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b355a6-e878-4640-be0a-08beeb6158a8 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:44.779930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b355a6-e878-4640-be0a-08beeb6158a8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:44.779930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b355a6-e878-4640-be0a-08beeb6158a8 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:44.781347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 69/150] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:44.787235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec45e6-48b6-40ba-9cb5-cb2a1c0dc659 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:44.787521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec45e6-48b6-40ba-9cb5-cb2a1c0dc659 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:44.787800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec45e6-48b6-40ba-9cb5-cb2a1c0dc659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:44.814801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-71ec45e6-48b6-40ba-9cb5-cb2a1c0dc659 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:44.816003 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-71ec45e6-48b6-40ba-9cb5-cb2a1c0dc659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:44.862327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:44.929705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 82/151] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:44.952618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b355a6-e878-4640-be0a-08beeb6158a8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:45.038240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a3521-a7bc-4cd3-9571-11bcba21c609 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.038553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a3521-a7bc-4cd3-9571-11bcba21c609 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.038796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a3521-a7bc-4cd3-9571-11bcba21c609 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.040935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 83/152] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:45.046635 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a767a5-b473-4f22-a003-bea9105f1b22 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.046921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a767a5-b473-4f22-a003-bea9105f1b22 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.047196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a767a5-b473-4f22-a003-bea9105f1b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.062276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b355a6-e878-4640-be0a-08beeb6158a8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:45.085246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-91a767a5-b473-4f22-a003-bea9105f1b22 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:45.086391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-91a767a5-b473-4f22-a003-bea9105f1b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:45.100968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7952a-a951-4dce-8326-b6efc9b493b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:45.101848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7952a-a951-4dce-8326-b6efc9b493b2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.102167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7952a-a951-4dce-8326-b6efc9b493b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.102816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7952a-a951-4dce-8326-b6efc9b493b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.122279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7952a-a951-4dce-8326-b6efc9b493b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a3d7952a-a951-4dce-8326-b6efc9b493b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:45.133367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:45.134301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:43:45.135041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:45.135616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 70/153] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:45.141902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7952a-a951-4dce-8326-b6efc9b493b2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:45.203958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 84/154] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:45.211639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:45.236421 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a767a5-b473-4f22-a003-bea9105f1b22 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:45.286815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7952a-a951-4dce-8326-b6efc9b493b2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:45.325122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d78fa-c4b5-4059-a016-d6f496ad76ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:45.325977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d78fa-c4b5-4059-a016-d6f496ad76ea None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.326711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d78fa-c4b5-4059-a016-d6f496ad76ea None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.327083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d78fa-c4b5-4059-a016-d6f496ad76ea None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.328231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d78fa-c4b5-4059-a016-d6f496ad76ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:45.328468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d78fa-c4b5-4059-a016-d6f496ad76ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:45.329977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d78fa-c4b5-4059-a016-d6f496ad76ea None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:45.341392 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 71/155] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:45.347046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d78fa-c4b5-4059-a016-d6f496ad76ea None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:45.360685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:45.369508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a767a5-b473-4f22-a003-bea9105f1b22 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:45.407659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a0f72-cab7-4a37-98c7-d0db431c9506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:45.408763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a0f72-cab7-4a37-98c7-d0db431c9506 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.409204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a0f72-cab7-4a37-98c7-d0db431c9506 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.409618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a0f72-cab7-4a37-98c7-d0db431c9506 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.422175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a0f72-cab7-4a37-98c7-d0db431c9506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1c8a0f72-cab7-4a37-98c7-d0db431c9506 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:45.430564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:45.431696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:45.432579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:45.435137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 85/156] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:43:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:43:45.440965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a0f72-cab7-4a37-98c7-d0db431c9506 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:45.459443 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d78fa-c4b5-4059-a016-d6f496ad76ea None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:45.469342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:45.498644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16347c4b-7147-4896-b72a-84a17f67c492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:45.499643 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16347c4b-7147-4896-b72a-84a17f67c492 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.500141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16347c4b-7147-4896-b72a-84a17f67c492 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.500332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16347c4b-7147-4896-b72a-84a17f67c492 None admin] PATH_INFO: `/v3/users/dacf5941777f42f28adcb47d5e613b75` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.501752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16347c4b-7147-4896-b72a-84a17f67c492 None admin] RBAC: Authorizing `identity:get_user(user_id=dacf5941777f42f28adcb47d5e613b75)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:45.503420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16347c4b-7147-4896-b72a-84a17f67c492 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:45.505476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 72/157] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:45 2022] GET /identity/v3/users/dacf5941777f42f28adcb47d5e613b75 => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:45.511604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16347c4b-7147-4896-b72a-84a17f67c492 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:45.556525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a0f72-cab7-4a37-98c7-d0db431c9506 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:45.593644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3461e9-32c6-48c6-900e-3b98a24468a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:45.594672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3461e9-32c6-48c6-900e-3b98a24468a7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.595158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3461e9-32c6-48c6-900e-3b98a24468a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.595670 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3461e9-32c6-48c6-900e-3b98a24468a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.596650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3461e9-32c6-48c6-900e-3b98a24468a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:45.597165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3461e9-32c6-48c6-900e-3b98a24468a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:45.598884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3461e9-32c6-48c6-900e-3b98a24468a7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:45.608730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 86/158] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:45.610516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3797e1-86ed-4b4a-b20f-6810b0691438 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.611411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3797e1-86ed-4b4a-b20f-6810b0691438 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.611982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3797e1-86ed-4b4a-b20f-6810b0691438 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.614141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 87/159] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:45.615224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3797e1-86ed-4b4a-b20f-6810b0691438 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:45.621797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16347c4b-7147-4896-b72a-84a17f67c492 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:45.636968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:45.661667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc070995-09dd-4a75-a74a-951de1bd62c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:45.662602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc070995-09dd-4a75-a74a-951de1bd62c2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.662884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc070995-09dd-4a75-a74a-951de1bd62c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.663155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc070995-09dd-4a75-a74a-951de1bd62c2 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.664433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc070995-09dd-4a75-a74a-951de1bd62c2 None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:45.666005 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc070995-09dd-4a75-a74a-951de1bd62c2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:45.668140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 73/160] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:45 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:45.669711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe107b0-421b-4c8d-99ae-00f5c9c19520 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.670133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe107b0-421b-4c8d-99ae-00f5c9c19520 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.670558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe107b0-421b-4c8d-99ae-00f5c9c19520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.690867 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-efe107b0-421b-4c8d-99ae-00f5c9c19520 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:45.692037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-efe107b0-421b-4c8d-99ae-00f5c9c19520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:45.743960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3797e1-86ed-4b4a-b20f-6810b0691438 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:45.744600 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:45.776711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 74/161] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:45.778057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f544b95-d48c-45a2-8e87-9c9f3ac55e6b None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.778345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f544b95-d48c-45a2-8e87-9c9f3ac55e6b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.778618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f544b95-d48c-45a2-8e87-9c9f3ac55e6b None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.783734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 75/162] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:45.783734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f544b95-d48c-45a2-8e87-9c9f3ac55e6b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:45.794769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f771a-531a-48b5-9ad9-790e73026048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:45.795773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f771a-531a-48b5-9ad9-790e73026048 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.796060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f771a-531a-48b5-9ad9-790e73026048 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.796407 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f771a-531a-48b5-9ad9-790e73026048 None admin] PATH_INFO: `/v3/users/5fc8fd4688454d8c8596f77d99812999` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.797714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f771a-531a-48b5-9ad9-790e73026048 None admin] RBAC: Authorizing `identity:get_user(user_id=5fc8fd4688454d8c8596f77d99812999)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:45.799281 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f771a-531a-48b5-9ad9-790e73026048 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:45.801347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 88/163] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:45 2022] GET /identity/v3/users/5fc8fd4688454d8c8596f77d99812999 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:45.802521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbd7e0-1b49-4d17-b782-6cca51e9ca41 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.802912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbd7e0-1b49-4d17-b782-6cca51e9ca41 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.803192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbd7e0-1b49-4d17-b782-6cca51e9ca41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.826650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d1dbd7e0-1b49-4d17-b782-6cca51e9ca41 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:45.829401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d1dbd7e0-1b49-4d17-b782-6cca51e9ca41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:45.905337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f544b95-d48c-45a2-8e87-9c9f3ac55e6b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:45.929579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 89/164] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:45.930536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbd7e0-1b49-4d17-b782-6cca51e9ca41 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:45.948783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:45.957541 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22456749-8486-4b7e-b470-21d861445315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:45.958460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22456749-8486-4b7e-b470-21d861445315 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:45.958738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22456749-8486-4b7e-b470-21d861445315 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:45.959008 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22456749-8486-4b7e-b470-21d861445315 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3/users/dacf5941777f42f28adcb47d5e613b75/roles/989e97787f6c43b5aa2603cc1b732f40` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:45.961059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22456749-8486-4b7e-b470-21d861445315 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad2772943334037bb33953f59d816e3, user_id=dacf5941777f42f28adcb47d5e613b75, role_id=989e97787f6c43b5aa2603cc1b732f40)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:45.962517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22456749-8486-4b7e-b470-21d861445315 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:45.985423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 76/165] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:43:45 2022] PUT /identity/v3/projects/3ad2772943334037bb33953f59d816e3/users/dacf5941777f42f28adcb47d5e613b75/roles/989e97787f6c43b5aa2603cc1b732f40 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:43:45.986481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22456749-8486-4b7e-b470-21d861445315 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:46.024880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:46.048008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbd7e0-1b49-4d17-b782-6cca51e9ca41 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.086370 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be7764-d697-4559-9027-1e0fddad5db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.087312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be7764-d697-4559-9027-1e0fddad5db9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.087594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be7764-d697-4559-9027-1e0fddad5db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.087867 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be7764-d697-4559-9027-1e0fddad5db9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:46.100061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22456749-8486-4b7e-b470-21d861445315 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.100295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be7764-d697-4559-9027-1e0fddad5db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-86be7764-d697-4559-9027-1e0fddad5db9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:46.108933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:46.109903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:46.110653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 90/166] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:43:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:46.111544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be7764-d697-4559-9027-1e0fddad5db9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:46.138807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32024ca8-605e-4a5c-8650-733fd989d41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.139595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32024ca8-605e-4a5c-8650-733fd989d41f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.139855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32024ca8-605e-4a5c-8650-733fd989d41f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.140142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32024ca8-605e-4a5c-8650-733fd989d41f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:46.194701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:46.218114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32024ca8-605e-4a5c-8650-733fd989d41f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:46.220059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32024ca8-605e-4a5c-8650-733fd989d41f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:46.222503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be7764-d697-4559-9027-1e0fddad5db9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.256916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:46.262299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79bab9-82cf-4e17-bac4-1d606dec310e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.263135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79bab9-82cf-4e17-bac4-1d606dec310e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.263416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79bab9-82cf-4e17-bac4-1d606dec310e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.263695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79bab9-82cf-4e17-bac4-1d606dec310e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:46.276448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79bab9-82cf-4e17-bac4-1d606dec310e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9a79bab9-82cf-4e17-bac4-1d606dec310e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:46.284724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:46.285861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 91/167] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:46.286851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79bab9-82cf-4e17-bac4-1d606dec310e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:46.337504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 77/168] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:46.338752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32024ca8-605e-4a5c-8650-733fd989d41f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:46.403281 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79bab9-82cf-4e17-bac4-1d606dec310e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.439918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8b3de-b97c-47dd-9935-2600b10b9954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.440642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8b3de-b97c-47dd-9935-2600b10b9954 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.441121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8b3de-b97c-47dd-9935-2600b10b9954 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.441291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8b3de-b97c-47dd-9935-2600b10b9954 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:46.442077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8b3de-b97c-47dd-9935-2600b10b9954 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:46.442414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8b3de-b97c-47dd-9935-2600b10b9954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:46.443860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8b3de-b97c-47dd-9935-2600b10b9954 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:46.445625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:46.452075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32024ca8-605e-4a5c-8650-733fd989d41f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.453961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 92/169] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:43:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:46.455038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8b3de-b97c-47dd-9935-2600b10b9954 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:46.488333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cbeaa-8a59-4675-883c-43bff61725de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.489281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cbeaa-8a59-4675-883c-43bff61725de None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.489691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cbeaa-8a59-4675-883c-43bff61725de None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.490114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cbeaa-8a59-4675-883c-43bff61725de None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:46.491110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cbeaa-8a59-4675-883c-43bff61725de None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:46.491645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cbeaa-8a59-4675-883c-43bff61725de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:46.493632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cbeaa-8a59-4675-883c-43bff61725de None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:46.496269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:46.503732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 78/170] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:46.504802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cbeaa-8a59-4675-883c-43bff61725de None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:46.568935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8b3de-b97c-47dd-9935-2600b10b9954 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.606918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925e819-939b-4cd4-89bf-9d80288aece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.608138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925e819-939b-4cd4-89bf-9d80288aece0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.608560 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925e819-939b-4cd4-89bf-9d80288aece0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.608994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925e819-939b-4cd4-89bf-9d80288aece0 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:46.609996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cbeaa-8a59-4675-883c-43bff61725de None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.610487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925e819-939b-4cd4-89bf-9d80288aece0 None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:46.612314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925e819-939b-4cd4-89bf-9d80288aece0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:46.614613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 93/171] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:46 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:46.615626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925e819-939b-4cd4-89bf-9d80288aece0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:46.646416 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5df1a2-fa9d-41e6-ae9a-a1102ed3fe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.647335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5df1a2-fa9d-41e6-ae9a-a1102ed3fe03 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.647585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5df1a2-fa9d-41e6-ae9a-a1102ed3fe03 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.647858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5df1a2-fa9d-41e6-ae9a-a1102ed3fe03 None admin] PATH_INFO: `/v3/users/0c0d3071ebef4f658f2da266190c1597` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:46.649208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5df1a2-fa9d-41e6-ae9a-a1102ed3fe03 None admin] RBAC: Authorizing `identity:get_user(user_id=0c0d3071ebef4f658f2da266190c1597)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:46.650773 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5df1a2-fa9d-41e6-ae9a-a1102ed3fe03 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:46.652461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 79/172] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:46 2022] GET /identity/v3/users/0c0d3071ebef4f658f2da266190c1597 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:46.653452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5df1a2-fa9d-41e6-ae9a-a1102ed3fe03 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:46.684511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:46.721788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:46.726328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925e819-939b-4cd4-89bf-9d80288aece0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.761734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5df1a2-fa9d-41e6-ae9a-a1102ed3fe03 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.763775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed72b3-3c08-4d1a-bc1f-145c7692a8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.764953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed72b3-3c08-4d1a-bc1f-145c7692a8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.765393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed72b3-3c08-4d1a-bc1f-145c7692a8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.765801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed72b3-3c08-4d1a-bc1f-145c7692a8a3 None admin] PATH_INFO: `/v3/users/0c0d3071ebef4f658f2da266190c1597` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:46.767154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed72b3-3c08-4d1a-bc1f-145c7692a8a3 None admin] RBAC: Authorizing `identity:get_user(user_id=0c0d3071ebef4f658f2da266190c1597)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:46.768816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed72b3-3c08-4d1a-bc1f-145c7692a8a3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:46.770675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 94/173] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:46 2022] GET /identity/v3/users/0c0d3071ebef4f658f2da266190c1597 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:46.771674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed72b3-3c08-4d1a-bc1f-145c7692a8a3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:46.797332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e136c97-0b41-442f-a108-cb4c0fba3ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.798002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e136c97-0b41-442f-a108-cb4c0fba3ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.798283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e136c97-0b41-442f-a108-cb4c0fba3ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.798554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e136c97-0b41-442f-a108-cb4c0fba3ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:46.799397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e136c97-0b41-442f-a108-cb4c0fba3ad1 None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=3ad2772943334037bb33953f59d816e3, user.id=5fc8fd4688454d8c8596f77d99812999) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:46.799704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e136c97-0b41-442f-a108-cb4c0fba3ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:46.801299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e136c97-0b41-442f-a108-cb4c0fba3ad1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:46.808216 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 80/174] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:46 2022] GET /identity/v3/role_assignments?user.id=5fc8fd4688454d8c8596f77d99812999&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=60bd1c44ce154338a7face8949d07281 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:46.808957 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e136c97-0b41-442f-a108-cb4c0fba3ad1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:46.890527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed72b3-3c08-4d1a-bc1f-145c7692a8a3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.918682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e136c97-0b41-442f-a108-cb4c0fba3ad1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:46.923966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:46.929260 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be610551-2a50-473f-aefe-fb328d4e76a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.930242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be610551-2a50-473f-aefe-fb328d4e76a3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.930668 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be610551-2a50-473f-aefe-fb328d4e76a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.931083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be610551-2a50-473f-aefe-fb328d4e76a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:46.947660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:46.956093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e81a8-8e21-4f27-912f-81467a042874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:46.956780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e81a8-8e21-4f27-912f-81467a042874 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:46.957064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e81a8-8e21-4f27-912f-81467a042874 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:46.957644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e81a8-8e21-4f27-912f-81467a042874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:47.017555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be610551-2a50-473f-aefe-fb328d4e76a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:47.025633 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be610551-2a50-473f-aefe-fb328d4e76a3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:47.034056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e81a8-8e21-4f27-912f-81467a042874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:47.035539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e81a8-8e21-4f27-912f-81467a042874 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:47.198418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 95/175] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:47.204223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be610551-2a50-473f-aefe-fb328d4e76a3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:47.252588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:47.271087 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 81/176] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:47.294240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e81a8-8e21-4f27-912f-81467a042874 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:47.319158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be610551-2a50-473f-aefe-fb328d4e76a3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:47.359935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38636c-48ed-4e53-9e62-90c898934435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:47.361524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38636c-48ed-4e53-9e62-90c898934435 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:47.362000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38636c-48ed-4e53-9e62-90c898934435 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:47.362486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38636c-48ed-4e53-9e62-90c898934435 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:47.363992 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38636c-48ed-4e53-9e62-90c898934435 None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:47.365792 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38636c-48ed-4e53-9e62-90c898934435 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:47.368081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 96/177] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:47 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:47.375840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38636c-48ed-4e53-9e62-90c898934435 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:47.377029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:47.475284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e81a8-8e21-4f27-912f-81467a042874 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:47.529925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38636c-48ed-4e53-9e62-90c898934435 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:47.532468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:47.537217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f7964-ae20-4004-ad8e-5cdca155f796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:47.538113 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f7964-ae20-4004-ad8e-5cdca155f796 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:47.538387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f7964-ae20-4004-ad8e-5cdca155f796 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:47.538663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f7964-ae20-4004-ad8e-5cdca155f796 None admin] PATH_INFO: `/v3/projects/eddb8e2d61d24035bd48e7c45500ea0f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:47.539932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f7964-ae20-4004-ad8e-5cdca155f796 None admin] RBAC: Authorizing `identity:get_project(project_id=eddb8e2d61d24035bd48e7c45500ea0f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:47.541516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f7964-ae20-4004-ad8e-5cdca155f796 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:47.543415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 82/178] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:47 2022] GET /identity/v3/projects/eddb8e2d61d24035bd48e7c45500ea0f => generated 312 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:47.549814 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f7964-ae20-4004-ad8e-5cdca155f796 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:47.597628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab62f1-0937-456e-b3cc-cfa300ff4851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:47.598296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab62f1-0937-456e-b3cc-cfa300ff4851 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:47.598576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab62f1-0937-456e-b3cc-cfa300ff4851 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:47.598854 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab62f1-0937-456e-b3cc-cfa300ff4851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:47.599793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab62f1-0937-456e-b3cc-cfa300ff4851 None admin] RBAC: Adding query filter params (role.id=e2c9006dc77f473f8e1064f285400a98, scope.project.id=3ad2772943334037bb33953f59d816e3, user.id=0c0d3071ebef4f658f2da266190c1597) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:47.600118 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab62f1-0937-456e-b3cc-cfa300ff4851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:47.601844 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab62f1-0937-456e-b3cc-cfa300ff4851 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:47.623491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 97/179] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:47 2022] GET /identity/v3/role_assignments?user.id=0c0d3071ebef4f658f2da266190c1597&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=e2c9006dc77f473f8e1064f285400a98 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:47.663166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:47.669603 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f7964-ae20-4004-ad8e-5cdca155f796 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:47.716339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1667c1e4-6eab-427c-bd30-aa647dd849e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:47.717043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667c1e4-6eab-427c-bd30-aa647dd849e0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:47.717318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667c1e4-6eab-427c-bd30-aa647dd849e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:47.717591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667c1e4-6eab-427c-bd30-aa647dd849e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:47.718476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1667c1e4-6eab-427c-bd30-aa647dd849e0 None admin] RBAC: Adding query filter params (role.id=1403cb7e4e264c68b63e467a2e433699, scope.project.id=eddb8e2d61d24035bd48e7c45500ea0f, user.id=0c0d3071ebef4f658f2da266190c1597) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:47.718790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1667c1e4-6eab-427c-bd30-aa647dd849e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:47.720350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1667c1e4-6eab-427c-bd30-aa647dd849e0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:47.727606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 83/180] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:47 2022] GET /identity/v3/role_assignments?user.id=0c0d3071ebef4f658f2da266190c1597&scope.project.id=eddb8e2d61d24035bd48e7c45500ea0f&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:48.471505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f9a65-8b9b-4dc8-a751-1cfc1d556427 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:48.471882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f9a65-8b9b-4dc8-a751-1cfc1d556427 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:48.472055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f9a65-8b9b-4dc8-a751-1cfc1d556427 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:48.473499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 98/181] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:48.478696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcbd20-fb4e-40a5-b569-261caec24e12 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:48.479141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcbd20-fb4e-40a5-b569-261caec24e12 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:48.479308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcbd20-fb4e-40a5-b569-261caec24e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:48.496415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cebcbd20-fb4e-40a5-b569-261caec24e12 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:48.497617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cebcbd20-fb4e-40a5-b569-261caec24e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:48.569036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 84/182] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:48.592286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f9a65-8b9b-4dc8-a751-1cfc1d556427 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:48.696987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f9a65-8b9b-4dc8-a751-1cfc1d556427 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:48.717509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13857d0f-f8ae-436f-9ac1-312a9c405613 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:48.717847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13857d0f-f8ae-436f-9ac1-312a9c405613 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:48.718060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13857d0f-f8ae-436f-9ac1-312a9c405613 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:48.719472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 85/183] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:48.724736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81b710-9533-4f60-a2c6-8659da2c55a4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:48.725242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81b710-9533-4f60-a2c6-8659da2c55a4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:48.725708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81b710-9533-4f60-a2c6-8659da2c55a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:48.732663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3c880-7c03-4f15-9e93-797a3d35c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:48.733366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c880-7c03-4f15-9e93-797a3d35c7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:48.733646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c880-7c03-4f15-9e93-797a3d35c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:48.733916 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c880-7c03-4f15-9e93-797a3d35c7a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:48.743745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-fb81b710-9533-4f60-a2c6-8659da2c55a4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:48.744747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3c880-7c03-4f15-9e93-797a3d35c7a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:48.745336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fb81b710-9533-4f60-a2c6-8659da2c55a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5ab3c880-7c03-4f15-9e93-797a3d35c7a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:48.752522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:48.753590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:48.754587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 99/184] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:43:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:48.755828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb60c6a-92f7-4457-b96a-4972accdfa8f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:48.755828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb60c6a-92f7-4457-b96a-4972accdfa8f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:48.755828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb60c6a-92f7-4457-b96a-4972accdfa8f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:48.757297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 100/185] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:48.761254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb60c6a-92f7-4457-b96a-4972accdfa8f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:48.787994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:48.836419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 86/186] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:43:48.837824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eb2c4-c956-44fc-a38a-c9a555cfd99e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:48.838117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eb2c4-c956-44fc-a38a-c9a555cfd99e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:48.838399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eb2c4-c956-44fc-a38a-c9a555cfd99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:48.862567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-662eb2c4-c956-44fc-a38a-c9a555cfd99e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:48.864820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-662eb2c4-c956-44fc-a38a-c9a555cfd99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:48.877244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb60c6a-92f7-4457-b96a-4972accdfa8f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:48.880932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:48.913829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93c3da-3889-4bf1-b4e4-c18b5886cc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:48.914551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93c3da-3889-4bf1-b4e4-c18b5886cc04 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:48.914833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93c3da-3889-4bf1-b4e4-c18b5886cc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:48.915106 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93c3da-3889-4bf1-b4e4-c18b5886cc04 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:48.915863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93c3da-3889-4bf1-b4e4-c18b5886cc04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:48.916196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93c3da-3889-4bf1-b4e4-c18b5886cc04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:48.917595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93c3da-3889-4bf1-b4e4-c18b5886cc04 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:48.930719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 101/187] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:48.930719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93c3da-3889-4bf1-b4e4-c18b5886cc04 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:48.941698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 87/188] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:48.942453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662eb2c4-c956-44fc-a38a-c9a555cfd99e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:49.032173 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:49.051854 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93c3da-3889-4bf1-b4e4-c18b5886cc04 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:49.052922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662eb2c4-c956-44fc-a38a-c9a555cfd99e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:49.089664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f7d8-23d7-4b71-997c-00f095719f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:49.089869 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46067e7c-ee63-4846-9435-8e36d3b25b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:49.090491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46067e7c-ee63-4846-9435-8e36d3b25b1f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.090617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f7d8-23d7-4b71-997c-00f095719f78 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.090788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46067e7c-ee63-4846-9435-8e36d3b25b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.090880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f7d8-23d7-4b71-997c-00f095719f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.091037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46067e7c-ee63-4846-9435-8e36d3b25b1f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.091127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f7d8-23d7-4b71-997c-00f095719f78 None admin] PATH_INFO: `/v3/users/74fbb60cb909462dbf684f1a0a7ded26` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.102567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46067e7c-ee63-4846-9435-8e36d3b25b1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:49.104621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2f7d8-23d7-4b71-997c-00f095719f78 None admin] RBAC: Authorizing `identity:get_user(user_id=74fbb60cb909462dbf684f1a0a7ded26)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:49.106118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2f7d8-23d7-4b71-997c-00f095719f78 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:49.108001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 88/189] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:48 2022] GET /identity/v3/users/74fbb60cb909462dbf684f1a0a7ded26 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:49.108708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f7d8-23d7-4b71-997c-00f095719f78 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-46067e7c-ee63-4846-9435-8e36d3b25b1f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:49.115073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:49.115909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:49.117015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 102/190] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:43:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:49.117670 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46067e7c-ee63-4846-9435-8e36d3b25b1f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:49.124473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:49.217855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f7d8-23d7-4b71-997c-00f095719f78 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:49.221028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46067e7c-ee63-4846-9435-8e36d3b25b1f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:49.251619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e311d48-72bb-4397-82db-1a6841fb0309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:49.252471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e311d48-72bb-4397-82db-1a6841fb0309 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.252755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e311d48-72bb-4397-82db-1a6841fb0309 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.253028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e311d48-72bb-4397-82db-1a6841fb0309 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.253116 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9f394-6a21-4aa7-8a64-65279c783658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:49.253762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9f394-6a21-4aa7-8a64-65279c783658 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.254046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9f394-6a21-4aa7-8a64-65279c783658 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.254271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e311d48-72bb-4397-82db-1a6841fb0309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:49.254383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9f394-6a21-4aa7-8a64-65279c783658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.255733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e311d48-72bb-4397-82db-1a6841fb0309 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:49.257408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 89/191] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:49.258239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e311d48-72bb-4397-82db-1a6841fb0309 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:49.270086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:49.321699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9f394-6a21-4aa7-8a64-65279c783658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:49.323160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9f394-6a21-4aa7-8a64-65279c783658 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:49.339270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:49.372065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e311d48-72bb-4397-82db-1a6841fb0309 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:49.416479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b5fc-3b9d-4495-94a3-e9c137b26716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:49.417471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b5fc-3b9d-4495-94a3-e9c137b26716 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.417887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b5fc-3b9d-4495-94a3-e9c137b26716 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.418290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b5fc-3b9d-4495-94a3-e9c137b26716 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.419467 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43b5fc-3b9d-4495-94a3-e9c137b26716 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:49.419931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43b5fc-3b9d-4495-94a3-e9c137b26716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:49.422176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43b5fc-3b9d-4495-94a3-e9c137b26716 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:49.426563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 103/192] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:49.428053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d924162-d15b-4a01-b8d3-388abfb93aae None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.428369 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d924162-d15b-4a01-b8d3-388abfb93aae None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.428650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d924162-d15b-4a01-b8d3-388abfb93aae None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.430303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 104/193] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:49.431505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d924162-d15b-4a01-b8d3-388abfb93aae None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:49.435032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 90/194] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:43:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:49.435032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b5fc-3b9d-4495-94a3-e9c137b26716 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:49.487003 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:49.538201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b5fc-3b9d-4495-94a3-e9c137b26716 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:49.542429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d924162-d15b-4a01-b8d3-388abfb93aae None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:49.570563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bac99f-1003-43df-b6c8-4a120ab144df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:49.571430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bac99f-1003-43df-b6c8-4a120ab144df None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.571760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bac99f-1003-43df-b6c8-4a120ab144df None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.571970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bac99f-1003-43df-b6c8-4a120ab144df None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.573178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bac99f-1003-43df-b6c8-4a120ab144df None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:49.574644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bac99f-1003-43df-b6c8-4a120ab144df None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:49.576749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 91/195] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:49 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:49.576883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3711d9-94a2-4684-a58c-ac2c8d584519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:49.577865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d2c24-2476-4533-bb9b-71aeb7f80af4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.578144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d2c24-2476-4533-bb9b-71aeb7f80af4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.578412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d2c24-2476-4533-bb9b-71aeb7f80af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.580867 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3711d9-94a2-4684-a58c-ac2c8d584519 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.581157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3711d9-94a2-4684-a58c-ac2c8d584519 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.581702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3711d9-94a2-4684-a58c-ac2c8d584519 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.582152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3711d9-94a2-4684-a58c-ac2c8d584519 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:49.583656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3711d9-94a2-4684-a58c-ac2c8d584519 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:49.589532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:49.595275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cc2d2c24-2476-4533-bb9b-71aeb7f80af4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:49.596516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cc2d2c24-2476-4533-bb9b-71aeb7f80af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:49.603606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 105/196] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:43:49 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:49.604632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3711d9-94a2-4684-a58c-ac2c8d584519 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:49.703918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 92/197] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:49.703918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d2c24-2476-4533-bb9b-71aeb7f80af4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:49.720569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3711d9-94a2-4684-a58c-ac2c8d584519 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:49.723534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:49.762876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72d0bb-a9a0-4443-bd82-1174006f7769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:49.763773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72d0bb-a9a0-4443-bd82-1174006f7769 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.764051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72d0bb-a9a0-4443-bd82-1174006f7769 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.764352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72d0bb-a9a0-4443-bd82-1174006f7769 None admin] PATH_INFO: `/v3/users/74fbb60cb909462dbf684f1a0a7ded26` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.765568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72d0bb-a9a0-4443-bd82-1174006f7769 None admin] RBAC: Authorizing `identity:get_user(user_id=74fbb60cb909462dbf684f1a0a7ded26)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:49.767053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72d0bb-a9a0-4443-bd82-1174006f7769 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:49.769252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 106/198] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:49 2022] GET /identity/v3/users/74fbb60cb909462dbf684f1a0a7ded26 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:49.772198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72d0bb-a9a0-4443-bd82-1174006f7769 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:49.840928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d2c24-2476-4533-bb9b-71aeb7f80af4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:49.890139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2c07a-6998-443d-8c89-c51eee5dc3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:49.890437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72d0bb-a9a0-4443-bd82-1174006f7769 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:49.890817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2c07a-6998-443d-8c89-c51eee5dc3fe None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.891098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2c07a-6998-443d-8c89-c51eee5dc3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.891374 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2c07a-6998-443d-8c89-c51eee5dc3fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.892541 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2c07a-6998-443d-8c89-c51eee5dc3fe None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=8c32c58c2df149f4842efa86dea7e18f, user.id=74fbb60cb909462dbf684f1a0a7ded26) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:49.892853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2c07a-6998-443d-8c89-c51eee5dc3fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:49.895310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2c07a-6998-443d-8c89-c51eee5dc3fe None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:49.898715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:49.904027 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 93/199] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:49 2022] GET /identity/v3/role_assignments?user.id=74fbb60cb909462dbf684f1a0a7ded26&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=60bd1c44ce154338a7face8949d07281 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:49.905067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2c07a-6998-443d-8c89-c51eee5dc3fe None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:49.942276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1ba8-f9b5-4b17-8f8d-88dfae3efc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:49.943649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1ba8-f9b5-4b17-8f8d-88dfae3efc69 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:49.944422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1ba8-f9b5-4b17-8f8d-88dfae3efc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:49.944653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1ba8-f9b5-4b17-8f8d-88dfae3efc69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:49.963081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e1ba8-f9b5-4b17-8f8d-88dfae3efc69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-2b7e1ba8-f9b5-4b17-8f8d-88dfae3efc69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:49.974705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:49.975645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:43:49.976500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:49.984544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 107/200] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:49.987541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1ba8-f9b5-4b17-8f8d-88dfae3efc69 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:50.051163 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:50.076004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2c07a-6998-443d-8c89-c51eee5dc3fe None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:50.141976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1ba8-f9b5-4b17-8f8d-88dfae3efc69 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:50.176057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4df80-e1ab-4de0-a198-b4dbb612a99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:50.177560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4df80-e1ab-4de0-a198-b4dbb612a99f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:50.177848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4df80-e1ab-4de0-a198-b4dbb612a99f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:50.178138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4df80-e1ab-4de0-a198-b4dbb612a99f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:50.180746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b46cb8-1cbc-4e90-b3e3-2ebddb685cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:50.181433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b46cb8-1cbc-4e90-b3e3-2ebddb685cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:50.181713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b46cb8-1cbc-4e90-b3e3-2ebddb685cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:50.181990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b46cb8-1cbc-4e90-b3e3-2ebddb685cc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:50.182760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b46cb8-1cbc-4e90-b3e3-2ebddb685cc6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:50.183066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b46cb8-1cbc-4e90-b3e3-2ebddb685cc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:50.184523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b46cb8-1cbc-4e90-b3e3-2ebddb685cc6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:50.204818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 108/201] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:50.212799 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b46cb8-1cbc-4e90-b3e3-2ebddb685cc6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:50.306992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:50.313950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4df80-e1ab-4de0-a198-b4dbb612a99f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:50.315677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4df80-e1ab-4de0-a198-b4dbb612a99f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:50.377268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:50.387360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b46cb8-1cbc-4e90-b3e3-2ebddb685cc6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:50.426104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c402a69-47b7-41c5-9612-5b6feedda75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:50.426920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c402a69-47b7-41c5-9612-5b6feedda75b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:50.427199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c402a69-47b7-41c5-9612-5b6feedda75b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:50.427474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c402a69-47b7-41c5-9612-5b6feedda75b None admin] PATH_INFO: `/v3/users/dacf5941777f42f28adcb47d5e613b75` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:50.428714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c402a69-47b7-41c5-9612-5b6feedda75b None admin] RBAC: Authorizing `identity:get_user(user_id=dacf5941777f42f28adcb47d5e613b75)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:50.430972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c402a69-47b7-41c5-9612-5b6feedda75b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:50.432890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 109/202] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:50 2022] GET /identity/v3/users/dacf5941777f42f28adcb47d5e613b75 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:50.438062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c402a69-47b7-41c5-9612-5b6feedda75b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:50.525827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 94/203] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:50.529881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4df80-e1ab-4de0-a198-b4dbb612a99f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:50.642286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c402a69-47b7-41c5-9612-5b6feedda75b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:50.664157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:50.667930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4df80-e1ab-4de0-a198-b4dbb612a99f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:50.705904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8222413-a118-4009-9c6f-c74cbb5b6864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:50.707070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8222413-a118-4009-9c6f-c74cbb5b6864 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:50.707519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8222413-a118-4009-9c6f-c74cbb5b6864 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:50.707974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8222413-a118-4009-9c6f-c74cbb5b6864 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:50.709520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8222413-a118-4009-9c6f-c74cbb5b6864 None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:50.711363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8222413-a118-4009-9c6f-c74cbb5b6864 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:50.713630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 95/204] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:50 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:50.716415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04959af-c3a1-47cd-b351-3f1a55139493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:50.717291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04959af-c3a1-47cd-b351-3f1a55139493 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:50.717734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04959af-c3a1-47cd-b351-3f1a55139493 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:50.718168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04959af-c3a1-47cd-b351-3f1a55139493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:50.719110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8222413-a118-4009-9c6f-c74cbb5b6864 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:50.758986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:50.803823 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04959af-c3a1-47cd-b351-3f1a55139493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:50.806082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04959af-c3a1-47cd-b351-3f1a55139493 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:50.956410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8222413-a118-4009-9c6f-c74cbb5b6864 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:50.992267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299c82f-338d-4b54-a6ac-8e0cbd8b5436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:50.993303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299c82f-338d-4b54-a6ac-8e0cbd8b5436 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:50.993774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299c82f-338d-4b54-a6ac-8e0cbd8b5436 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:50.994211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299c82f-338d-4b54-a6ac-8e0cbd8b5436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:50.994745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 110/205] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 20:43:50.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299c82f-338d-4b54-a6ac-8e0cbd8b5436 None admin] RBAC: Adding query filter params (role.id=e2c9006dc77f473f8e1064f285400a98, scope.project.id=8c32c58c2df149f4842efa86dea7e18f, user.id=74fbb60cb909462dbf684f1a0a7ded26) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:51.000204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299c82f-338d-4b54-a6ac-8e0cbd8b5436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:51.001953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299c82f-338d-4b54-a6ac-8e0cbd8b5436 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:51.008886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 96/206] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:50 2022] GET /identity/v3/role_assignments?user.id=74fbb60cb909462dbf684f1a0a7ded26&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=e2c9006dc77f473f8e1064f285400a98 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:51.018654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04959af-c3a1-47cd-b351-3f1a55139493 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:51.158984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:51.220379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04959af-c3a1-47cd-b351-3f1a55139493 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:51.265758 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01270d39-56a5-44b2-9b4f-a79b6553bdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:51.266644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01270d39-56a5-44b2-9b4f-a79b6553bdbd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:51.266920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01270d39-56a5-44b2-9b4f-a79b6553bdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:51.267195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01270d39-56a5-44b2-9b4f-a79b6553bdbd None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:51.268426 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01270d39-56a5-44b2-9b4f-a79b6553bdbd None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:51.269906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01270d39-56a5-44b2-9b4f-a79b6553bdbd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:51.271952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 111/207] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:51 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:51.278446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299c82f-338d-4b54-a6ac-8e0cbd8b5436 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:51.337272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:51.430998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299c82f-338d-4b54-a6ac-8e0cbd8b5436 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:51.476185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ee756-af97-4e83-91a4-9c93a374c7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:51.477412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ee756-af97-4e83-91a4-9c93a374c7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:51.477887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ee756-af97-4e83-91a4-9c93a374c7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:51.478348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ee756-af97-4e83-91a4-9c93a374c7a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:51.479419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ee756-af97-4e83-91a4-9c93a374c7a3 None admin] RBAC: Adding query filter params (role.id=989e97787f6c43b5aa2603cc1b732f40, scope.project.id=3ad2772943334037bb33953f59d816e3, user.id=dacf5941777f42f28adcb47d5e613b75) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:51.479935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ee756-af97-4e83-91a4-9c93a374c7a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:51.481607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ee756-af97-4e83-91a4-9c93a374c7a3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:51.488452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 97/208] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:51 2022] GET /identity/v3/role_assignments?user.id=dacf5941777f42f28adcb47d5e613b75&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=989e97787f6c43b5aa2603cc1b732f40 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:52.030787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58646ced-bc68-433c-b5d5-96f8d31da4c0 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.031321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58646ced-bc68-433c-b5d5-96f8d31da4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.031321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58646ced-bc68-433c-b5d5-96f8d31da4c0 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.033014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 112/209] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:52.038525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c5936-115c-48d9-b234-5d7dd1383b40 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.039078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c5936-115c-48d9-b234-5d7dd1383b40 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.039150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c5936-115c-48d9-b234-5d7dd1383b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.059441 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9e1c5936-115c-48d9-b234-5d7dd1383b40 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:52.060697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9e1c5936-115c-48d9-b234-5d7dd1383b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:52.140319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 98/210] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:52.147020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7ba26-dad9-4442-84c1-bb1392f6c902 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.147523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7ba26-dad9-4442-84c1-bb1392f6c902 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.148024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7ba26-dad9-4442-84c1-bb1392f6c902 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.149720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 113/211] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:52.155103 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27489470-b6c2-4fbb-85be-102d2c097472 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.155393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27489470-b6c2-4fbb-85be-102d2c097472 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.155673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27489470-b6c2-4fbb-85be-102d2c097472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.166248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7ba26-dad9-4442-84c1-bb1392f6c902 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:52.172665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-27489470-b6c2-4fbb-85be-102d2c097472 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:52.173843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-27489470-b6c2-4fbb-85be-102d2c097472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:52.194521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:52.244156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 99/212] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:52.244156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b85127-54e6-4a18-b26b-100870ee36d4 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.244156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b85127-54e6-4a18-b26b-100870ee36d4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.244670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b85127-54e6-4a18-b26b-100870ee36d4 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.245554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 100/213] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:52.253740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4cdd7-f7c6-4da0-993d-e9afec096a60 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.254183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4cdd7-f7c6-4da0-993d-e9afec096a60 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.254613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4cdd7-f7c6-4da0-993d-e9afec096a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.273972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-85a4cdd7-f7c6-4da0-993d-e9afec096a60 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:52.275171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-85a4cdd7-f7c6-4da0-993d-e9afec096a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:52.283885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7ba26-dad9-4442-84c1-bb1392f6c902 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:52.311360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:52.319984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3d8f6-6c37-4b25-98ae-0144faebafc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:52.320876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3d8f6-6c37-4b25-98ae-0144faebafc4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.321155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3d8f6-6c37-4b25-98ae-0144faebafc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.321430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3d8f6-6c37-4b25-98ae-0144faebafc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.332465 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3d8f6-6c37-4b25-98ae-0144faebafc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f5e3d8f6-6c37-4b25-98ae-0144faebafc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:52.340776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:52.341654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:52.342429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 114/214] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:43:52.343070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3d8f6-6c37-4b25-98ae-0144faebafc4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:52.351781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 101/215] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:43:52.352928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4cdd7-f7c6-4da0-993d-e9afec096a60 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:52.450338 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:52.469529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4cdd7-f7c6-4da0-993d-e9afec096a60 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:52.476579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3d8f6-6c37-4b25-98ae-0144faebafc4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:52.505868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1272c-c90b-4e78-a637-565c249b5dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:52.506541 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1272c-c90b-4e78-a637-565c249b5dbf None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.506820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1272c-c90b-4e78-a637-565c249b5dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.507093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1272c-c90b-4e78-a637-565c249b5dbf None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.507929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1272c-c90b-4e78-a637-565c249b5dbf None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:52.508268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1272c-c90b-4e78-a637-565c249b5dbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:52.509675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1272c-c90b-4e78-a637-565c249b5dbf None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:52.513143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fac132-7d2b-4115-ad56-c70784b14e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:52.514137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fac132-7d2b-4115-ad56-c70784b14e35 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.514580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fac132-7d2b-4115-ad56-c70784b14e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.515028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fac132-7d2b-4115-ad56-c70784b14e35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.516218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fac132-7d2b-4115-ad56-c70784b14e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:52.517660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fac132-7d2b-4115-ad56-c70784b14e35 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:52.518549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 102/216] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:52.519397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 115/217] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:52.523933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1272c-c90b-4e78-a637-565c249b5dbf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:52.524751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fac132-7d2b-4115-ad56-c70784b14e35 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:52.584172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:52.631746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fac132-7d2b-4115-ad56-c70784b14e35 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:52.633306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1272c-c90b-4e78-a637-565c249b5dbf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:52.666612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2617b67-b760-4392-81df-2cc890c52b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:52.666783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6d59f-3387-4896-badd-e043e3b8950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:52.667435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6d59f-3387-4896-badd-e043e3b8950e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.667532 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2617b67-b760-4392-81df-2cc890c52b53 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.667718 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6d59f-3387-4896-badd-e043e3b8950e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.667810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2617b67-b760-4392-81df-2cc890c52b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.667993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6d59f-3387-4896-badd-e043e3b8950e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.668093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2617b67-b760-4392-81df-2cc890c52b53 None admin] PATH_INFO: `/v3/users/e3027ca9016748f8b94fabea5db19d34` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.678797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6d59f-3387-4896-badd-e043e3b8950e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:52.683959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2617b67-b760-4392-81df-2cc890c52b53 None admin] RBAC: Authorizing `identity:get_user(user_id=e3027ca9016748f8b94fabea5db19d34)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:52.685471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2617b67-b760-4392-81df-2cc890c52b53 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-95c6d59f-3387-4896-badd-e043e3b8950e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:52.686580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:52.687413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:52.688233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:52.688831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 116/218] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:52 2022] GET /identity/v3/users/e3027ca9016748f8b94fabea5db19d34 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:52.688895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 103/219] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:43:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:52.692558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2617b67-b760-4392-81df-2cc890c52b53 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:52.693138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6d59f-3387-4896-badd-e043e3b8950e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:52.708379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:52.796750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2617b67-b760-4392-81df-2cc890c52b53 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:52.806742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6d59f-3387-4896-badd-e043e3b8950e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:52.812439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:52.839601 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35878d08-a5b5-40f0-bb53-d7f33fd1d8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:52.840303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35878d08-a5b5-40f0-bb53-d7f33fd1d8ce None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.840585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35878d08-a5b5-40f0-bb53-d7f33fd1d8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.840846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35878d08-a5b5-40f0-bb53-d7f33fd1d8ce None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.841770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35878d08-a5b5-40f0-bb53-d7f33fd1d8ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:52.843228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35878d08-a5b5-40f0-bb53-d7f33fd1d8ce None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:52.847677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619f56-9a36-4ce0-97f9-33d2bc20deb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:52.848449 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619f56-9a36-4ce0-97f9-33d2bc20deb5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:52.848798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619f56-9a36-4ce0-97f9-33d2bc20deb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:52.849140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619f56-9a36-4ce0-97f9-33d2bc20deb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:52.866613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 117/220] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:43:52 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:52.873315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35878d08-a5b5-40f0-bb53-d7f33fd1d8ce None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:52.927700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d619f56-9a36-4ce0-97f9-33d2bc20deb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:52.930058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d619f56-9a36-4ce0-97f9-33d2bc20deb5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:52.941701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:53.018823 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35878d08-a5b5-40f0-bb53-d7f33fd1d8ce None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:53.042093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 104/221] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:53.048608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619f56-9a36-4ce0-97f9-33d2bc20deb5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:53.068677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe24d66-4dbc-4a92-b27a-c56319206f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:53.069375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe24d66-4dbc-4a92-b27a-c56319206f7b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:53.069634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe24d66-4dbc-4a92-b27a-c56319206f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:53.069920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe24d66-4dbc-4a92-b27a-c56319206f7b None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:53.070737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe24d66-4dbc-4a92-b27a-c56319206f7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:53.071058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe24d66-4dbc-4a92-b27a-c56319206f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:53.072491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe24d66-4dbc-4a92-b27a-c56319206f7b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:53.083751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 118/222] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:53.089352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe24d66-4dbc-4a92-b27a-c56319206f7b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:53.127463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:53.174832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619f56-9a36-4ce0-97f9-33d2bc20deb5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:53.185469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:53.220150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe24d66-4dbc-4a92-b27a-c56319206f7b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:53.221394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eed517-447a-4f82-b4d0-5aa9dc5cdeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:53.222954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed517-447a-4f82-b4d0-5aa9dc5cdeca None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:53.222954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed517-447a-4f82-b4d0-5aa9dc5cdeca None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:53.223204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed517-447a-4f82-b4d0-5aa9dc5cdeca None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:53.225419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eed517-447a-4f82-b4d0-5aa9dc5cdeca None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:53.227040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eed517-447a-4f82-b4d0-5aa9dc5cdeca None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:53.229212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 105/223] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:53 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:53.237109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eed517-447a-4f82-b4d0-5aa9dc5cdeca None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:53.270058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895dfbd-108c-4cc9-8ece-553de98764c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:53.271203 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895dfbd-108c-4cc9-8ece-553de98764c7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:53.271677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895dfbd-108c-4cc9-8ece-553de98764c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:53.272196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895dfbd-108c-4cc9-8ece-553de98764c7 None admin] PATH_INFO: `/v3/users/74fbb60cb909462dbf684f1a0a7ded26` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:53.273663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895dfbd-108c-4cc9-8ece-553de98764c7 None admin] RBAC: Authorizing `identity:get_user(user_id=74fbb60cb909462dbf684f1a0a7ded26)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:53.275347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895dfbd-108c-4cc9-8ece-553de98764c7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:53.277294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 119/224] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:53 2022] GET /identity/v3/users/74fbb60cb909462dbf684f1a0a7ded26 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:53.282939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895dfbd-108c-4cc9-8ece-553de98764c7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:53.364022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eed517-447a-4f82-b4d0-5aa9dc5cdeca None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:53.405819 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:53.409049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895dfbd-108c-4cc9-8ece-553de98764c7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:53.410430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5079ffc-89ed-4a30-87f2-bca370ea7bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:53.412151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5079ffc-89ed-4a30-87f2-bca370ea7bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:53.413263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5079ffc-89ed-4a30-87f2-bca370ea7bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:53.413548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5079ffc-89ed-4a30-87f2-bca370ea7bf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:53.414563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5079ffc-89ed-4a30-87f2-bca370ea7bf8 None admin] RBAC: Adding query filter params (role.id=1403cb7e4e264c68b63e467a2e433699, scope.project.id=8c32c58c2df149f4842efa86dea7e18f, user.id=e3027ca9016748f8b94fabea5db19d34) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:53.414877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5079ffc-89ed-4a30-87f2-bca370ea7bf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:53.416388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5079ffc-89ed-4a30-87f2-bca370ea7bf8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:53.428189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 106/225] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:53 2022] GET /identity/v3/role_assignments?user.id=e3027ca9016748f8b94fabea5db19d34&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:53.430048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0412f7-25ab-485e-ab38-fc6f43256b61 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:53.430679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0412f7-25ab-485e-ab38-fc6f43256b61 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:53.431136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0412f7-25ab-485e-ab38-fc6f43256b61 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:53.434101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 107/226] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:53.441534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b4e38-c2d8-4295-8d19-165513171b85 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:53.442186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b4e38-c2d8-4295-8d19-165513171b85 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:53.444015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b4e38-c2d8-4295-8d19-165513171b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:53.476376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-099b4e38-c2d8-4295-8d19-165513171b85 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:53.478847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-099b4e38-c2d8-4295-8d19-165513171b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:53.479716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f29f0f-88e5-4d3a-803c-ebfac4fbc926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:53.480956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f29f0f-88e5-4d3a-803c-ebfac4fbc926 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:53.481439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f29f0f-88e5-4d3a-803c-ebfac4fbc926 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:53.481894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f29f0f-88e5-4d3a-803c-ebfac4fbc926 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:53.483849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f29f0f-88e5-4d3a-803c-ebfac4fbc926 None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:53.487660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f29f0f-88e5-4d3a-803c-ebfac4fbc926 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:53.490286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 120/227] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:53 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:53.498364 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f29f0f-88e5-4d3a-803c-ebfac4fbc926 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:53.561202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:53.618521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 108/228] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:53.654974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b4e38-c2d8-4295-8d19-165513171b85 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:53.661752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f29f0f-88e5-4d3a-803c-ebfac4fbc926 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:53.702991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3afb7-4ba5-413f-aa77-2d0888403ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:53.704285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3afb7-4ba5-413f-aa77-2d0888403ef3 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:53.704761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3afb7-4ba5-413f-aa77-2d0888403ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:53.705226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3afb7-4ba5-413f-aa77-2d0888403ef3 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f/users/74fbb60cb909462dbf684f1a0a7ded26/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:53.707749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3afb7-4ba5-413f-aa77-2d0888403ef3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c32c58c2df149f4842efa86dea7e18f, user_id=74fbb60cb909462dbf684f1a0a7ded26, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:53.709605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3afb7-4ba5-413f-aa77-2d0888403ef3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:53.734406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 121/229] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:43:53 2022] PUT /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f/users/74fbb60cb909462dbf684f1a0a7ded26/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:43:53.802983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b4e38-c2d8-4295-8d19-165513171b85 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:53.841192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b193b04-016d-47d5-a94e-0cddbc7f870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:53.842064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b193b04-016d-47d5-a94e-0cddbc7f870f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:53.842344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b193b04-016d-47d5-a94e-0cddbc7f870f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:53.842617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b193b04-016d-47d5-a94e-0cddbc7f870f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:53.854663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b193b04-016d-47d5-a94e-0cddbc7f870f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-7b193b04-016d-47d5-a94e-0cddbc7f870f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:53.863240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:53.864135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:53.864919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 109/230] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:43:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:43:53.873348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3afb7-4ba5-413f-aa77-2d0888403ef3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:53.914231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:54.002828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3afb7-4ba5-413f-aa77-2d0888403ef3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:54.040991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68890e-d802-493b-aeb7-274d4dd81186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:54.042098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68890e-d802-493b-aeb7-274d4dd81186 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:54.042587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68890e-d802-493b-aeb7-274d4dd81186 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:54.043046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68890e-d802-493b-aeb7-274d4dd81186 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:54.044098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68890e-d802-493b-aeb7-274d4dd81186 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:54.044631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68890e-d802-493b-aeb7-274d4dd81186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:54.046296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68890e-d802-493b-aeb7-274d4dd81186 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:54.055564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 122/231] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:43:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:54.061290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b193b04-016d-47d5-a94e-0cddbc7f870f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:54.108463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:54.199106 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b193b04-016d-47d5-a94e-0cddbc7f870f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:54.253515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c726a9-6399-4148-bebc-55a8a8c9b26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:54.254766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c726a9-6399-4148-bebc-55a8a8c9b26a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:54.255260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c726a9-6399-4148-bebc-55a8a8c9b26a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:54.255745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c726a9-6399-4148-bebc-55a8a8c9b26a None admin] PATH_INFO: `/v3/users/74fbb60cb909462dbf684f1a0a7ded26` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:54.257181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c726a9-6399-4148-bebc-55a8a8c9b26a None admin] RBAC: Authorizing `identity:get_user(user_id=74fbb60cb909462dbf684f1a0a7ded26)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:54.258875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c726a9-6399-4148-bebc-55a8a8c9b26a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:54.260794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 110/232] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:54 2022] GET /identity/v3/users/74fbb60cb909462dbf684f1a0a7ded26 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:54.266518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68890e-d802-493b-aeb7-274d4dd81186 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:54.396064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68890e-d802-493b-aeb7-274d4dd81186 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:54.404417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:54.443201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dc4d0-02a1-4c35-901c-1e1859b2eb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:54.444090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc4d0-02a1-4c35-901c-1e1859b2eb34 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:54.444405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc4d0-02a1-4c35-901c-1e1859b2eb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:54.444680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc4d0-02a1-4c35-901c-1e1859b2eb34 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:54.445944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21dc4d0-02a1-4c35-901c-1e1859b2eb34 None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:54.447536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21dc4d0-02a1-4c35-901c-1e1859b2eb34 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:54.449325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 123/233] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:54 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:54.455914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c726a9-6399-4148-bebc-55a8a8c9b26a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:54.593622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:54.597345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c726a9-6399-4148-bebc-55a8a8c9b26a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:54.650817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3036baa-d962-48f8-b24f-dc5a53978df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:54.651654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3036baa-d962-48f8-b24f-dc5a53978df5 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:54.651935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3036baa-d962-48f8-b24f-dc5a53978df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:54.652235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3036baa-d962-48f8-b24f-dc5a53978df5 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f/users/74fbb60cb909462dbf684f1a0a7ded26/roles/e2c9006dc77f473f8e1064f285400a98` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:54.654490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3036baa-d962-48f8-b24f-dc5a53978df5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c32c58c2df149f4842efa86dea7e18f, user_id=74fbb60cb909462dbf684f1a0a7ded26, role_id=e2c9006dc77f473f8e1064f285400a98)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:54.655938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3036baa-d962-48f8-b24f-dc5a53978df5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:54.694818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 111/234] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:43:54 2022] PUT /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f/users/74fbb60cb909462dbf684f1a0a7ded26/roles/e2c9006dc77f473f8e1064f285400a98 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:43:55.552263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087084f-3105-46aa-9fd4-3452cb5e92bf None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:55.552825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087084f-3105-46aa-9fd4-3452cb5e92bf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:55.552825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087084f-3105-46aa-9fd4-3452cb5e92bf None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:55.554499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 124/235] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:55.559670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643317c3-629e-442a-856d-2ef7dd960e6a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:55.560069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643317c3-629e-442a-856d-2ef7dd960e6a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:55.560277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643317c3-629e-442a-856d-2ef7dd960e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:55.580880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-643317c3-629e-442a-856d-2ef7dd960e6a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:55.582055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-643317c3-629e-442a-856d-2ef7dd960e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:55.588368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026a1b-f154-406b-8192-5b3484cfbfb2 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:55.588658 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026a1b-f154-406b-8192-5b3484cfbfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:55.588941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026a1b-f154-406b-8192-5b3484cfbfb2 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:55.590447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 125/236] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:43:55.601383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62f537-b25b-43ed-a2f8-6c62680f1bef None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:55.601709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62f537-b25b-43ed-a2f8-6c62680f1bef None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:55.601986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62f537-b25b-43ed-a2f8-6c62680f1bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:55.621298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-dc62f537-b25b-43ed-a2f8-6c62680f1bef None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:55.622489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dc62f537-b25b-43ed-a2f8-6c62680f1bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:55.687300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 112/237] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:43:55.707266 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 126/238] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:43:55.717878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643317c3-629e-442a-856d-2ef7dd960e6a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:55.754699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62f537-b25b-43ed-a2f8-6c62680f1bef None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:55.824220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:55.825127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643317c3-629e-442a-856d-2ef7dd960e6a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:55.886127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1408b-25d9-4c0c-985f-de6656c73c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:55.887503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1408b-25d9-4c0c-985f-de6656c73c3d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:55.887790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1408b-25d9-4c0c-985f-de6656c73c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:55.888061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1408b-25d9-4c0c-985f-de6656c73c3d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:55.901605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1408b-25d9-4c0c-985f-de6656c73c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:55.910562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-36f1408b-25d9-4c0c-985f-de6656c73c3d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:55.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:55.914457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:55.915672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 113/239] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:55.916353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d93472-01fe-47ed-a757-20cc3d54cc01 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:55.916353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d93472-01fe-47ed-a757-20cc3d54cc01 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:55.916353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d93472-01fe-47ed-a757-20cc3d54cc01 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:55.918954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 114/240] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:55.919660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d93472-01fe-47ed-a757-20cc3d54cc01 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:55.928885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62f537-b25b-43ed-a2f8-6c62680f1bef None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:55.969257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0295b-43bc-41b6-b9e3-e80eeeb86367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:55.970423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0295b-43bc-41b6-b9e3-e80eeeb86367 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:55.970918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0295b-43bc-41b6-b9e3-e80eeeb86367 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:55.971417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0295b-43bc-41b6-b9e3-e80eeeb86367 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:55.972799 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0295b-43bc-41b6-b9e3-e80eeeb86367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:55.974521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0295b-43bc-41b6-b9e3-e80eeeb86367 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:55.981243 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 127/241] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:55.982829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06cd46-2817-4259-a819-38f4270f95f7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:55.983329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06cd46-2817-4259-a819-38f4270f95f7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:55.983838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06cd46-2817-4259-a819-38f4270f95f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.001124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9f06cd46-2817-4259-a819-38f4270f95f7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:56.002667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9f06cd46-2817-4259-a819-38f4270f95f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:56.049105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d93472-01fe-47ed-a757-20cc3d54cc01 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:56.072478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 128/242] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:56.073740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06cd46-2817-4259-a819-38f4270f95f7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:56.095804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61181ccd-75f7-4f22-abd4-c4e1b45e9175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:56.097748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181ccd-75f7-4f22-abd4-c4e1b45e9175 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:56.098494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181ccd-75f7-4f22-abd4-c4e1b45e9175 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:56.098775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181ccd-75f7-4f22-abd4-c4e1b45e9175 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.099606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61181ccd-75f7-4f22-abd4-c4e1b45e9175 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:56.099920 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61181ccd-75f7-4f22-abd4-c4e1b45e9175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:56.101471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61181ccd-75f7-4f22-abd4-c4e1b45e9175 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:56.111452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 115/243] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:56.112388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61181ccd-75f7-4f22-abd4-c4e1b45e9175 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:56.125855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:56.185253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:56.211319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06cd46-2817-4259-a819-38f4270f95f7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:56.234997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61181ccd-75f7-4f22-abd4-c4e1b45e9175 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:56.247337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374832a2-d8a1-4ee6-a983-be13fa31e990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:56.248119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374832a2-d8a1-4ee6-a983-be13fa31e990 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:56.248507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374832a2-d8a1-4ee6-a983-be13fa31e990 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:56.248851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374832a2-d8a1-4ee6-a983-be13fa31e990 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.249641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374832a2-d8a1-4ee6-a983-be13fa31e990 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:56.251174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374832a2-d8a1-4ee6-a983-be13fa31e990 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:56.275750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 129/244] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:43:56 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:43:56.281042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374832a2-d8a1-4ee6-a983-be13fa31e990 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:56.289784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22956b-d0d7-405a-a569-26c50a3be3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:56.291047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22956b-d0d7-405a-a569-26c50a3be3de None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:56.291561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22956b-d0d7-405a-a569-26c50a3be3de None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:56.292016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22956b-d0d7-405a-a569-26c50a3be3de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.308417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22956b-d0d7-405a-a569-26c50a3be3de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-ac22956b-d0d7-405a-a569-26c50a3be3de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:56.322526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:56.323509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:56.324332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:56.327973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 116/245] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:56.330515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d72c8d-539d-4f00-89f9-54754536ed9c None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:56.331013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d72c8d-539d-4f00-89f9-54754536ed9c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:56.331472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d72c8d-539d-4f00-89f9-54754536ed9c None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.333520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 117/246] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:56.335228 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d72c8d-539d-4f00-89f9-54754536ed9c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:56.425224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374832a2-d8a1-4ee6-a983-be13fa31e990 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:56.427602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:56.463751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6edf39-ddc0-4679-8827-d1df439145cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:56.465031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6edf39-ddc0-4679-8827-d1df439145cc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:56.465239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d72c8d-539d-4f00-89f9-54754536ed9c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:56.465761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6edf39-ddc0-4679-8827-d1df439145cc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:56.466218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6edf39-ddc0-4679-8827-d1df439145cc None admin] PATH_INFO: `/v3/users/69bcd53574694476a5f8a2e76839ca0a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.478899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:56.483452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6edf39-ddc0-4679-8827-d1df439145cc None admin] RBAC: Authorizing `identity:get_user(user_id=69bcd53574694476a5f8a2e76839ca0a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:56.486209 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6edf39-ddc0-4679-8827-d1df439145cc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:56.487509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 130/247] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:56 2022] GET /identity/v3/users/69bcd53574694476a5f8a2e76839ca0a => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:56.489296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ad7f-b2d3-4c2a-b97a-5bdfd64edde0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:56.489909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ad7f-b2d3-4c2a-b97a-5bdfd64edde0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:56.490475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ad7f-b2d3-4c2a-b97a-5bdfd64edde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.511781 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9d41ad7f-b2d3-4c2a-b97a-5bdfd64edde0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:56.513103 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9d41ad7f-b2d3-4c2a-b97a-5bdfd64edde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:56.515306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51013029-b1a8-4418-8d3c-588818fe31a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:56.516209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51013029-b1a8-4418-8d3c-588818fe31a4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:56.516618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51013029-b1a8-4418-8d3c-588818fe31a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:56.517020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51013029-b1a8-4418-8d3c-588818fe31a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.518055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51013029-b1a8-4418-8d3c-588818fe31a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:56.518562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51013029-b1a8-4418-8d3c-588818fe31a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:56.520182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51013029-b1a8-4418-8d3c-588818fe31a4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:56.531355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 118/248] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:56.532491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51013029-b1a8-4418-8d3c-588818fe31a4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:56.608712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 131/249] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:56.609739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41ad7f-b2d3-4c2a-b97a-5bdfd64edde0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:56.656872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51013029-b1a8-4418-8d3c-588818fe31a4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:56.698184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:56.707937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e33804-3ef9-4753-b898-7c9195933d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:56.709576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e33804-3ef9-4753-b898-7c9195933d8d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:56.709869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e33804-3ef9-4753-b898-7c9195933d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:56.710148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e33804-3ef9-4753-b898-7c9195933d8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.753712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41ad7f-b2d3-4c2a-b97a-5bdfd64edde0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:56.792454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:56.801527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b7d3a-3679-428d-832b-d6889c46bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:56.802871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7d3a-3679-428d-832b-d6889c46bbfb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:56.803345 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7d3a-3679-428d-832b-d6889c46bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:56.803862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7d3a-3679-428d-832b-d6889c46bbfb None admin] PATH_INFO: `/v3/users/e3027ca9016748f8b94fabea5db19d34` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.805371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b7d3a-3679-428d-832b-d6889c46bbfb None admin] RBAC: Authorizing `identity:get_user(user_id=e3027ca9016748f8b94fabea5db19d34)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:56.806111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e33804-3ef9-4753-b898-7c9195933d8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:56.807391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b7d3a-3679-428d-832b-d6889c46bbfb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:56.807599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e33804-3ef9-4753-b898-7c9195933d8d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:56.809635 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 132/250] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:56 2022] GET /identity/v3/users/e3027ca9016748f8b94fabea5db19d34 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:56.810595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b7d3a-3679-428d-832b-d6889c46bbfb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:56.917470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 119/251] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:56.918384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e33804-3ef9-4753-b898-7c9195933d8d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:56.931270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b7d3a-3679-428d-832b-d6889c46bbfb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:56.949560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:56.971015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0db73-c2c9-4816-92b8-dfc1106bb84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:56.972226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0db73-c2c9-4816-92b8-dfc1106bb84c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:56.972727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0db73-c2c9-4816-92b8-dfc1106bb84c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:56.973285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0db73-c2c9-4816-92b8-dfc1106bb84c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:56.985224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0db73-c2c9-4816-92b8-dfc1106bb84c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-17f0db73-c2c9-4816-92b8-dfc1106bb84c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:56.993827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:56.994717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:43:56.995474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:56.998313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 133/252] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:43:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:56.999337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0db73-c2c9-4816-92b8-dfc1106bb84c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:57.023738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e33804-3ef9-4753-b898-7c9195933d8d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:57.063375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:57.073005 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d33eed-1fc6-48b1-9771-cc7ce8071f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:57.073902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d33eed-1fc6-48b1-9771-cc7ce8071f9d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:57.074183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d33eed-1fc6-48b1-9771-cc7ce8071f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:57.074455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d33eed-1fc6-48b1-9771-cc7ce8071f9d None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:57.080906 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d33eed-1fc6-48b1-9771-cc7ce8071f9d None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:57.082421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d33eed-1fc6-48b1-9771-cc7ce8071f9d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:57.084295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 120/253] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:56 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:57.085201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d33eed-1fc6-48b1-9771-cc7ce8071f9d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:57.112976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0db73-c2c9-4816-92b8-dfc1106bb84c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:57.149397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1905fe-cb31-4fb3-b520-6dd7979108c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:57.150641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1905fe-cb31-4fb3-b520-6dd7979108c8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:57.151140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1905fe-cb31-4fb3-b520-6dd7979108c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:57.151606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1905fe-cb31-4fb3-b520-6dd7979108c8 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:57.153103 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1905fe-cb31-4fb3-b520-6dd7979108c8 None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:57.154908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1905fe-cb31-4fb3-b520-6dd7979108c8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:57.157082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 134/254] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:56 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:57.158130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1905fe-cb31-4fb3-b520-6dd7979108c8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:57.272776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1905fe-cb31-4fb3-b520-6dd7979108c8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:57.284194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d33eed-1fc6-48b1-9771-cc7ce8071f9d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:57.287121 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:57.300699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:57.311442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99bab-7bf5-4701-a0ec-ffbb374fe058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:57.312729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99bab-7bf5-4701-a0ec-ffbb374fe058 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:57.313223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99bab-7bf5-4701-a0ec-ffbb374fe058 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:57.313700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99bab-7bf5-4701-a0ec-ffbb374fe058 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f/users/e3027ca9016748f8b94fabea5db19d34/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:57.315917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e99bab-7bf5-4701-a0ec-ffbb374fe058 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c32c58c2df149f4842efa86dea7e18f, user_id=e3027ca9016748f8b94fabea5db19d34, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:57.317610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e99bab-7bf5-4701-a0ec-ffbb374fe058 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:57.323167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63de19-119a-4e86-a978-9acde83ea970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:57.323914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63de19-119a-4e86-a978-9acde83ea970 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:57.324283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63de19-119a-4e86-a978-9acde83ea970 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:57.324940 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63de19-119a-4e86-a978-9acde83ea970 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:57.326288 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63de19-119a-4e86-a978-9acde83ea970 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:57.327194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63de19-119a-4e86-a978-9acde83ea970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:57.329544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63de19-119a-4e86-a978-9acde83ea970 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:57.334716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 135/255] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:43:57 2022] PUT /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f/users/e3027ca9016748f8b94fabea5db19d34/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:43:57.335946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99bab-7bf5-4701-a0ec-ffbb374fe058 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:57.353129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 121/256] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:57.357228 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eaabf-3471-4a49-9427-28393ab85091 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:57.357846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eaabf-3471-4a49-9427-28393ab85091 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:57.358365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eaabf-3471-4a49-9427-28393ab85091 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:57.360141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 122/257] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:57.361185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eaabf-3471-4a49-9427-28393ab85091 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:57.547496 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99bab-7bf5-4701-a0ec-ffbb374fe058 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:57.558204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eaabf-3471-4a49-9427-28393ab85091 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:57.592723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bd1f1-0366-44fc-8be5-1f785606f884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:57.593462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bd1f1-0366-44fc-8be5-1f785606f884 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:57.593665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bd1f1-0366-44fc-8be5-1f785606f884 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:57.593942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bd1f1-0366-44fc-8be5-1f785606f884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:57.594969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bd1f1-0366-44fc-8be5-1f785606f884 None admin] RBAC: Adding query filter params (role.id=989e97787f6c43b5aa2603cc1b732f40, scope.project.id=8c32c58c2df149f4842efa86dea7e18f, user.id=69bcd53574694476a5f8a2e76839ca0a) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:57.595073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bd1f1-0366-44fc-8be5-1f785606f884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:57.597674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bd1f1-0366-44fc-8be5-1f785606f884 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:57.603593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 136/258] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:57 2022] GET /identity/v3/role_assignments?user.id=69bcd53574694476a5f8a2e76839ca0a&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=989e97787f6c43b5aa2603cc1b732f40 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:57.605057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c150d-8211-4a46-96ae-09eae189ab38 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:57.605289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c150d-8211-4a46-96ae-09eae189ab38 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:57.605572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c150d-8211-4a46-96ae-09eae189ab38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:57.613825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257eb7-2bef-4234-8b48-a013a29679ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:57.615300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257eb7-2bef-4234-8b48-a013a29679ec None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:57.615300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257eb7-2bef-4234-8b48-a013a29679ec None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:57.615493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257eb7-2bef-4234-8b48-a013a29679ec None admin] PATH_INFO: `/v3/users/74fbb60cb909462dbf684f1a0a7ded26` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:57.622923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49257eb7-2bef-4234-8b48-a013a29679ec None admin] RBAC: Authorizing `identity:get_user(user_id=74fbb60cb909462dbf684f1a0a7ded26)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:57.622923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49257eb7-2bef-4234-8b48-a013a29679ec None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:57.629012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 123/259] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:57 2022] GET /identity/v3/users/74fbb60cb909462dbf684f1a0a7ded26 => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:57.634492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257eb7-2bef-4234-8b48-a013a29679ec None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:57.636913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a86c150d-8211-4a46-96ae-09eae189ab38 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:57.638581 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a86c150d-8211-4a46-96ae-09eae189ab38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:57.692685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:57.717117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:57.765826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 137/260] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:57.826931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257eb7-2bef-4234-8b48-a013a29679ec None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:57.853231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c150d-8211-4a46-96ae-09eae189ab38 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:57.885575 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d5ce4-dc9b-4f57-89e2-1a3040860b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:57.886243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d5ce4-dc9b-4f57-89e2-1a3040860b51 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:57.886524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d5ce4-dc9b-4f57-89e2-1a3040860b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:57.886795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d5ce4-dc9b-4f57-89e2-1a3040860b51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:57.983443 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c150d-8211-4a46-96ae-09eae189ab38 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:58.063140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fea11-cdf3-4dcd-bfac-c90232e519ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:58.063976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fea11-cdf3-4dcd-bfac-c90232e519ec None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:58.064280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fea11-cdf3-4dcd-bfac-c90232e519ec None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:58.064550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fea11-cdf3-4dcd-bfac-c90232e519ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:58.081985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d5ce4-dc9b-4f57-89e2-1a3040860b51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:58.084453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d5ce4-dc9b-4f57-89e2-1a3040860b51 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:58.093491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fea11-cdf3-4dcd-bfac-c90232e519ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-783fea11-cdf3-4dcd-bfac-c90232e519ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:58.108083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:58.108983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:43:58.109744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:43:58.120216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 138/261] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:43:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:43:58.121432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fea11-cdf3-4dcd-bfac-c90232e519ec None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:58.155225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:58.222160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:58.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 124/262] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:58.290194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d5ce4-dc9b-4f57-89e2-1a3040860b51 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:58.351932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fea11-cdf3-4dcd-bfac-c90232e519ec None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:58.385495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d7e9a-6e26-4457-ba76-edbf9cb0ec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:58.386181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d7e9a-6e26-4457-ba76-edbf9cb0ec36 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:58.386466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d7e9a-6e26-4457-ba76-edbf9cb0ec36 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:58.386745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d7e9a-6e26-4457-ba76-edbf9cb0ec36 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:58.387505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d7e9a-6e26-4457-ba76-edbf9cb0ec36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:58.387812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d7e9a-6e26-4457-ba76-edbf9cb0ec36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:58.392129 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d7e9a-6e26-4457-ba76-edbf9cb0ec36 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:58.397824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 139/263] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:43:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:58.404433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d7e9a-6e26-4457-ba76-edbf9cb0ec36 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:58.480157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d5ce4-dc9b-4f57-89e2-1a3040860b51 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:58.514379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:58.522359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c6f7d-26a6-4db1-80f8-63ac778c3797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:58.523694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c6f7d-26a6-4db1-80f8-63ac778c3797 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:58.524752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c6f7d-26a6-4db1-80f8-63ac778c3797 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:58.525278 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c6f7d-26a6-4db1-80f8-63ac778c3797 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:58.526723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c6f7d-26a6-4db1-80f8-63ac778c3797 None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:58.528415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c6f7d-26a6-4db1-80f8-63ac778c3797 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:58.531037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 125/264] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:58 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:58.537163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c6f7d-26a6-4db1-80f8-63ac778c3797 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:58.647986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d7e9a-6e26-4457-ba76-edbf9cb0ec36 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:58.649913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:58.654758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c6f7d-26a6-4db1-80f8-63ac778c3797 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:58.706067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71feb4b-0e55-4fe0-8d7d-06195e12fbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:58.708388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71feb4b-0e55-4fe0-8d7d-06195e12fbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:58.709720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71feb4b-0e55-4fe0-8d7d-06195e12fbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:58.710599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71feb4b-0e55-4fe0-8d7d-06195e12fbd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:58.712892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71feb4b-0e55-4fe0-8d7d-06195e12fbd4 None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=8c32c58c2df149f4842efa86dea7e18f, user.id=74fbb60cb909462dbf684f1a0a7ded26) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:58.714250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71feb4b-0e55-4fe0-8d7d-06195e12fbd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:58.715895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71feb4b-0e55-4fe0-8d7d-06195e12fbd4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:58.723827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee404af-2059-4f78-bbd2-4148fe3d554c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:58.725098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee404af-2059-4f78-bbd2-4148fe3d554c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:58.726442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee404af-2059-4f78-bbd2-4148fe3d554c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:58.727093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee404af-2059-4f78-bbd2-4148fe3d554c None admin] PATH_INFO: `/v3/users/74fbb60cb909462dbf684f1a0a7ded26` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:58.729864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 126/265] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:58 2022] GET /identity/v3/role_assignments?user.id=74fbb60cb909462dbf684f1a0a7ded26&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=60bd1c44ce154338a7face8949d07281 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:58.730729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee404af-2059-4f78-bbd2-4148fe3d554c None admin] RBAC: Authorizing `identity:get_user(user_id=74fbb60cb909462dbf684f1a0a7ded26)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:58.732445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee404af-2059-4f78-bbd2-4148fe3d554c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:58.735783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 140/266] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:58 2022] GET /identity/v3/users/74fbb60cb909462dbf684f1a0a7ded26 => generated 283 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:58.741316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71feb4b-0e55-4fe0-8d7d-06195e12fbd4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:58.825047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:58.887923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71feb4b-0e55-4fe0-8d7d-06195e12fbd4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:58.936178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68791abb-8ae4-4885-827b-e2dfc0757bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:58.937198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68791abb-8ae4-4885-827b-e2dfc0757bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:58.937666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68791abb-8ae4-4885-827b-e2dfc0757bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:58.938124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68791abb-8ae4-4885-827b-e2dfc0757bf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:59.028985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68791abb-8ae4-4885-827b-e2dfc0757bf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:59.030822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68791abb-8ae4-4885-827b-e2dfc0757bf3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:59.088897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac454fda-ab43-4b51-9751-a4e315842635 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:59.089218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac454fda-ab43-4b51-9751-a4e315842635 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:59.089485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac454fda-ab43-4b51-9751-a4e315842635 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:59.091172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 141/267] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:59.096158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea975008-7032-4103-9b04-3489b32f82cb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:59.096503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea975008-7032-4103-9b04-3489b32f82cb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:59.096795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea975008-7032-4103-9b04-3489b32f82cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:59.112543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:59.127577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ea975008-7032-4103-9b04-3489b32f82cb None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:59.128808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ea975008-7032-4103-9b04-3489b32f82cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:59.172017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 127/268] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:43:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:43:59.177896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68791abb-8ae4-4885-827b-e2dfc0757bf3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:59.291803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68791abb-8ae4-4885-827b-e2dfc0757bf3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:59.314500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 142/269] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:59.328863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6b932-d6c7-4fa2-b964-f18c241e660f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:59.329831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b932-d6c7-4fa2-b964-f18c241e660f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:59.330118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b932-d6c7-4fa2-b964-f18c241e660f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:59.330396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b932-d6c7-4fa2-b964-f18c241e660f None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:59.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6b932-d6c7-4fa2-b964-f18c241e660f None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:59.333368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6b932-d6c7-4fa2-b964-f18c241e660f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:59.335561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 128/270] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:43:59 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:59.342089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea975008-7032-4103-9b04-3489b32f82cb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:59.345905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6b932-d6c7-4fa2-b964-f18c241e660f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:59.404078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:59.457196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:59.460923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea975008-7032-4103-9b04-3489b32f82cb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:59.461631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6b932-d6c7-4fa2-b964-f18c241e660f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:59.497712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a3212-617a-452c-be54-9df44ecbe3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:59.498407 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3212-617a-452c-be54-9df44ecbe3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:59.498721 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3212-617a-452c-be54-9df44ecbe3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:59.498967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3212-617a-452c-be54-9df44ecbe3a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:59.499116 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac089692-329f-482b-84cb-424bffd3d010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:59.499809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a3212-617a-452c-be54-9df44ecbe3a4 None admin] RBAC: Adding query filter params (role.id=e2c9006dc77f473f8e1064f285400a98, scope.project.id=8c32c58c2df149f4842efa86dea7e18f, user.id=74fbb60cb909462dbf684f1a0a7ded26) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:59.500074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac089692-329f-482b-84cb-424bffd3d010 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:59.500192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a3212-617a-452c-be54-9df44ecbe3a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:59.500636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac089692-329f-482b-84cb-424bffd3d010 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:59.500974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac089692-329f-482b-84cb-424bffd3d010 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:59.501805 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a3212-617a-452c-be54-9df44ecbe3a4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:59.516135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 143/271] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:43:59 2022] GET /identity/v3/role_assignments?user.id=74fbb60cb909462dbf684f1a0a7ded26&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=e2c9006dc77f473f8e1064f285400a98 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:59.516135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3de32-e2a9-41c6-81e1-c3f9de8839a6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:59.516135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3de32-e2a9-41c6-81e1-c3f9de8839a6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:59.516135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3de32-e2a9-41c6-81e1-c3f9de8839a6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:59.516135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 144/272] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:43:59.518062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3264ff-1529-4f70-91fb-70a07ebdb772 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:59.518418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3264ff-1529-4f70-91fb-70a07ebdb772 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:59.518760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3264ff-1529-4f70-91fb-70a07ebdb772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:59.528846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac089692-329f-482b-84cb-424bffd3d010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:43:59.536700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bf3264ff-1529-4f70-91fb-70a07ebdb772 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-ac089692-329f-482b-84cb-424bffd3d010 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:43:59.536970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:43:59.538014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:43:59.539135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:43:59.544235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bf3264ff-1529-4f70-91fb-70a07ebdb772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:43:59.545446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 129/273] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:43:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:43:59.553208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac089692-329f-482b-84cb-424bffd3d010 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:59.661618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 145/274] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:43:59.706539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3264ff-1529-4f70-91fb-70a07ebdb772 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:59.737922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:59.750830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac089692-329f-482b-84cb-424bffd3d010 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:59.802874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f169e6-f4e7-47d9-82b5-e71da2f63ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:59.803490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f169e6-f4e7-47d9-82b5-e71da2f63ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:59.803775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f169e6-f4e7-47d9-82b5-e71da2f63ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:59.804186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f169e6-f4e7-47d9-82b5-e71da2f63ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:59.805641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f169e6-f4e7-47d9-82b5-e71da2f63ee1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:43:59.805958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f169e6-f4e7-47d9-82b5-e71da2f63ee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:59.808283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f169e6-f4e7-47d9-82b5-e71da2f63ee1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:59.821349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 130/275] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:43:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:43:59.827428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f169e6-f4e7-47d9-82b5-e71da2f63ee1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:59.840766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:43:59.849249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3264ff-1529-4f70-91fb-70a07ebdb772 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:43:59.910057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34186a7-1f1f-4f7f-b910-1b9eaa3ee0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:43:59.910934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34186a7-1f1f-4f7f-b910-1b9eaa3ee0dd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:43:59.911222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34186a7-1f1f-4f7f-b910-1b9eaa3ee0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:43:59.911601 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34186a7-1f1f-4f7f-b910-1b9eaa3ee0dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:43:59.912818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34186a7-1f1f-4f7f-b910-1b9eaa3ee0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:43:59.914390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34186a7-1f1f-4f7f-b910-1b9eaa3ee0dd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:43:59.916054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 146/276] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:43:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:43:59.925018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34186a7-1f1f-4f7f-b910-1b9eaa3ee0dd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:43:59.970753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f169e6-f4e7-47d9-82b5-e71da2f63ee1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:00.017024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6d5da-10e5-454d-a8f2-3ea1b1465c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:00.018309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6d5da-10e5-454d-a8f2-3ea1b1465c02 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.018785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6d5da-10e5-454d-a8f2-3ea1b1465c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.019671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6d5da-10e5-454d-a8f2-3ea1b1465c02 None admin] PATH_INFO: `/v3/users/93a59339da564756914fa6fe098f209c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.038869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6d5da-10e5-454d-a8f2-3ea1b1465c02 None admin] RBAC: Authorizing `identity:get_user(user_id=93a59339da564756914fa6fe098f209c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:00.041163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6d5da-10e5-454d-a8f2-3ea1b1465c02 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:00.043693 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 131/277] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:43:59 2022] GET /identity/v3/users/93a59339da564756914fa6fe098f209c => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:00.044632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34186a7-1f1f-4f7f-b910-1b9eaa3ee0dd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:00.052642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6d5da-10e5-454d-a8f2-3ea1b1465c02 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:00.063305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:00.087659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75720f62-230a-468d-8642-9721cb379c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:00.088512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75720f62-230a-468d-8642-9721cb379c31 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.088797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75720f62-230a-468d-8642-9721cb379c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.089104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75720f62-230a-468d-8642-9721cb379c31 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.090146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75720f62-230a-468d-8642-9721cb379c31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:00.091409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75720f62-230a-468d-8642-9721cb379c31 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:00.114454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 147/278] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:43:59 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:44:00.164445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6d5da-10e5-454d-a8f2-3ea1b1465c02 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:00.204674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b7021-cdf3-4711-b84b-818cdb9c0a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:00.205655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7021-cdf3-4711-b84b-818cdb9c0a00 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.206127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7021-cdf3-4711-b84b-818cdb9c0a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.206548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7021-cdf3-4711-b84b-818cdb9c0a00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.207599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b7021-cdf3-4711-b84b-818cdb9c0a00 None admin] RBAC: Adding query filter params (role.id=1403cb7e4e264c68b63e467a2e433699, user.id=93a59339da564756914fa6fe098f209c, scope.system=all) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:00.208049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b7021-cdf3-4711-b84b-818cdb9c0a00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:00.209749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b7021-cdf3-4711-b84b-818cdb9c0a00 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:00.216745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 132/279] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Jul 11 20:44:00 2022] GET /identity/v3/role_assignments?user.id=93a59339da564756914fa6fe098f209c&scope.system=all&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:00.222263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8f2c0-679a-4ed0-a62a-03d828abee29 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.222555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8f2c0-679a-4ed0-a62a-03d828abee29 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.222832 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8f2c0-679a-4ed0-a62a-03d828abee29 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.224759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 148/280] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:00.231770 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467f754-2e48-4f07-bff4-d1cf4424371f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.232380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467f754-2e48-4f07-bff4-d1cf4424371f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.232817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467f754-2e48-4f07-bff4-d1cf4424371f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.255800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9467f754-2e48-4f07-bff4-d1cf4424371f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:00.257175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9467f754-2e48-4f07-bff4-d1cf4424371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:00.339530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 133/281] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:00.362829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8f2c0-679a-4ed0-a62a-03d828abee29 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:00.385108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:00.416675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dce9d-2bc9-4e0c-88b6-35f9df25f736 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.416988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dce9d-2bc9-4e0c-88b6-35f9df25f736 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.417238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dce9d-2bc9-4e0c-88b6-35f9df25f736 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.418648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 134/282] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:44:00.424656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a46c9-92e9-485d-9458-f8d288e51d2c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.424931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a46c9-92e9-485d-9458-f8d288e51d2c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.425207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a46c9-92e9-485d-9458-f8d288e51d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.433578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:00.446789 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cd3a46c9-92e9-485d-9458-f8d288e51d2c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:00.448053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cd3a46c9-92e9-485d-9458-f8d288e51d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:00.493416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8f2c0-679a-4ed0-a62a-03d828abee29 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:00.535971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 135/283] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:00.542555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04723e5-ec12-4531-81cc-c7b8240cdf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:00.543377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04723e5-ec12-4531-81cc-c7b8240cdf53 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.543664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04723e5-ec12-4531-81cc-c7b8240cdf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.543941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04723e5-ec12-4531-81cc-c7b8240cdf53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.555826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04723e5-ec12-4531-81cc-c7b8240cdf53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c04723e5-ec12-4531-81cc-c7b8240cdf53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:00.563733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:00.564697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:00.565503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 149/284] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:00.572133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a46c9-92e9-485d-9458-f8d288e51d2c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:00.572288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04723e5-ec12-4531-81cc-c7b8240cdf53 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:00.658984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:00.673262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a46c9-92e9-485d-9458-f8d288e51d2c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:00.686094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04723e5-ec12-4531-81cc-c7b8240cdf53 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:00.710799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94b694-4fe8-4cf1-a6ff-43fbded2cd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:00.711885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94b694-4fe8-4cf1-a6ff-43fbded2cd51 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.711991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94b694-4fe8-4cf1-a6ff-43fbded2cd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.712317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94b694-4fe8-4cf1-a6ff-43fbded2cd51 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.722920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12092b2b-72d7-4a58-bdd9-1de2e978d6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:00.723361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94b694-4fe8-4cf1-a6ff-43fbded2cd51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:00.724011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12092b2b-72d7-4a58-bdd9-1de2e978d6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.724479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12092b2b-72d7-4a58-bdd9-1de2e978d6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.724911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12092b2b-72d7-4a58-bdd9-1de2e978d6a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.725853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12092b2b-72d7-4a58-bdd9-1de2e978d6a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:00.728304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12092b2b-72d7-4a58-bdd9-1de2e978d6a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:00.729020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12092b2b-72d7-4a58-bdd9-1de2e978d6a3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:00.729524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-fc94b694-4fe8-4cf1-a6ff-43fbded2cd51 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:00.737460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:00.738774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:00.739542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 136/285] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:00.749640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94b694-4fe8-4cf1-a6ff-43fbded2cd51 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:00.762828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 150/286] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:00.768216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12092b2b-72d7-4a58-bdd9-1de2e978d6a3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:00.911874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12092b2b-72d7-4a58-bdd9-1de2e978d6a3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:00.950490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26427d69-0236-4226-bc04-417d4cf61624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:00.951757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26427d69-0236-4226-bc04-417d4cf61624 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:00.952037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26427d69-0236-4226-bc04-417d4cf61624 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:00.952496 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26427d69-0236-4226-bc04-417d4cf61624 None admin] PATH_INFO: `/v3/users/e3027ca9016748f8b94fabea5db19d34` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:00.954043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26427d69-0236-4226-bc04-417d4cf61624 None admin] RBAC: Authorizing `identity:get_user(user_id=e3027ca9016748f8b94fabea5db19d34)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:00.955576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26427d69-0236-4226-bc04-417d4cf61624 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:00.958080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 151/287] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:44:00 2022] GET /identity/v3/users/e3027ca9016748f8b94fabea5db19d34 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:00.967091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26427d69-0236-4226-bc04-417d4cf61624 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:00.996524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:00.998345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94b694-4fe8-4cf1-a6ff-43fbded2cd51 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:01.088125 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26427d69-0236-4226-bc04-417d4cf61624 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:01.105364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77400-6253-4881-9164-f913250c185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:01.106055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77400-6253-4881-9164-f913250c185c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:01.106336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77400-6253-4881-9164-f913250c185c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:01.106611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77400-6253-4881-9164-f913250c185c None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:01.107381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b77400-6253-4881-9164-f913250c185c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:01.107689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b77400-6253-4881-9164-f913250c185c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:01.109135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b77400-6253-4881-9164-f913250c185c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:01.126478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 137/288] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:01.133663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77400-6253-4881-9164-f913250c185c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:01.144009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd3162-f73e-414d-9ba0-7c2fe10b4108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:01.144700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd3162-f73e-414d-9ba0-7c2fe10b4108 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:01.144980 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd3162-f73e-414d-9ba0-7c2fe10b4108 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:01.145250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd3162-f73e-414d-9ba0-7c2fe10b4108 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:01.254180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd3162-f73e-414d-9ba0-7c2fe10b4108 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:01.255836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd3162-f73e-414d-9ba0-7c2fe10b4108 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:01.267142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:01.269566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:01.341437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77400-6253-4881-9164-f913250c185c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:01.414194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61e90a-9dff-4432-833c-9cd929a0bdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:01.415084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61e90a-9dff-4432-833c-9cd929a0bdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:01.416075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61e90a-9dff-4432-833c-9cd929a0bdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:01.416571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61e90a-9dff-4432-833c-9cd929a0bdc3 None admin] PATH_INFO: `/v3/users/69bcd53574694476a5f8a2e76839ca0a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:01.419004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61e90a-9dff-4432-833c-9cd929a0bdc3 None admin] RBAC: Authorizing `identity:get_user(user_id=69bcd53574694476a5f8a2e76839ca0a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:01.420481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61e90a-9dff-4432-833c-9cd929a0bdc3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:01.422134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 138/289] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:44:01 2022] GET /identity/v3/users/69bcd53574694476a5f8a2e76839ca0a => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:01.428341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61e90a-9dff-4432-833c-9cd929a0bdc3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:01.462135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 152/290] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:01.466970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd3162-f73e-414d-9ba0-7c2fe10b4108 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:01.595384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61e90a-9dff-4432-833c-9cd929a0bdc3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:01.639672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:01.652265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd3162-f73e-414d-9ba0-7c2fe10b4108 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:01.663989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03f26b-1698-4593-b49a-425490075f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:01.665421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03f26b-1698-4593-b49a-425490075f21 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:01.665892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03f26b-1698-4593-b49a-425490075f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:01.666376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03f26b-1698-4593-b49a-425490075f21 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:01.667826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03f26b-1698-4593-b49a-425490075f21 None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:01.669538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03f26b-1698-4593-b49a-425490075f21 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:01.670300 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:01.671968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 139/291] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:01 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:01.681195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03f26b-1698-4593-b49a-425490075f21 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:01.726808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92887b23-f445-4105-8fdb-7ac267fa686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:01.727731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92887b23-f445-4105-8fdb-7ac267fa686d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:01.728015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92887b23-f445-4105-8fdb-7ac267fa686d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:01.728312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92887b23-f445-4105-8fdb-7ac267fa686d None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:01.730189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92887b23-f445-4105-8fdb-7ac267fa686d None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:01.731662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92887b23-f445-4105-8fdb-7ac267fa686d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:01.733570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 153/292] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:01 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:01.739540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92887b23-f445-4105-8fdb-7ac267fa686d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:01.845324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03f26b-1698-4593-b49a-425490075f21 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:01.882743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834f930-af34-42a2-9d2b-64c581a8e6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:01.883889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834f930-af34-42a2-9d2b-64c581a8e6f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:01.884356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834f930-af34-42a2-9d2b-64c581a8e6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:01.884790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834f930-af34-42a2-9d2b-64c581a8e6f6 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f/users/69bcd53574694476a5f8a2e76839ca0a/roles/989e97787f6c43b5aa2603cc1b732f40` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:01.885391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92887b23-f445-4105-8fdb-7ac267fa686d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:01.887115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834f930-af34-42a2-9d2b-64c581a8e6f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c32c58c2df149f4842efa86dea7e18f, user_id=69bcd53574694476a5f8a2e76839ca0a, role_id=989e97787f6c43b5aa2603cc1b732f40)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:01.888755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834f930-af34-42a2-9d2b-64c581a8e6f6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:01.904862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 140/293] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:44:01 2022] PUT /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f/users/69bcd53574694476a5f8a2e76839ca0a/roles/989e97787f6c43b5aa2603cc1b732f40 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:01.935928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252ddb1-9c2f-4676-a31a-f2ec6f6626a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:01.936696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252ddb1-9c2f-4676-a31a-f2ec6f6626a3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:01.936978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252ddb1-9c2f-4676-a31a-f2ec6f6626a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:01.937247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252ddb1-9c2f-4676-a31a-f2ec6f6626a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:01.938103 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252ddb1-9c2f-4676-a31a-f2ec6f6626a3 None admin] RBAC: Adding query filter params (role.id=1403cb7e4e264c68b63e467a2e433699, scope.project.id=8c32c58c2df149f4842efa86dea7e18f, user.id=e3027ca9016748f8b94fabea5db19d34) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:01.938408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252ddb1-9c2f-4676-a31a-f2ec6f6626a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:01.939899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252ddb1-9c2f-4676-a31a-f2ec6f6626a3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:01.952519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 154/294] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:01 2022] GET /identity/v3/role_assignments?user.id=e3027ca9016748f8b94fabea5db19d34&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:02.677309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1a2c-00b8-4270-921e-df65ec047dd0 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:02.678064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1a2c-00b8-4270-921e-df65ec047dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:02.678545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1a2c-00b8-4270-921e-df65ec047dd0 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:02.680921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 141/295] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:02.686429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c029fb-4216-402b-9966-f26a45440b48 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:02.686962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c029fb-4216-402b-9966-f26a45440b48 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:02.687496 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c029fb-4216-402b-9966-f26a45440b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:02.690455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd261489-0fe4-43b7-b420-708994b0c0a9 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:02.690744 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd261489-0fe4-43b7-b420-708994b0c0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:02.691019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd261489-0fe4-43b7-b420-708994b0c0a9 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:02.692644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 142/296] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:02.699045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869237c2-b5a0-4282-a57f-e3cab174d6c8 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:02.699466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869237c2-b5a0-4282-a57f-e3cab174d6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:02.699870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869237c2-b5a0-4282-a57f-e3cab174d6c8 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:02.702492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 143/297] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:02.704610 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe90024-4463-4de7-bc17-21106596844c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:02.705149 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe90024-4463-4de7-bc17-21106596844c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:02.705952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe90024-4463-4de7-bc17-21106596844c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:02.710642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-36c029fb-4216-402b-9966-f26a45440b48 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:02.711807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-36c029fb-4216-402b-9966-f26a45440b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:02.723442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bbe90024-4463-4de7-bc17-21106596844c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:02.724638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bbe90024-4463-4de7-bc17-21106596844c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:02.749547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:02.780536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:02.783353 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 155/298] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:02.784770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40121ce2-5666-4f74-ab95-d8a7c750707b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:02.785233 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40121ce2-5666-4f74-ab95-d8a7c750707b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:02.785510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40121ce2-5666-4f74-ab95-d8a7c750707b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:02.805143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-40121ce2-5666-4f74-ab95-d8a7c750707b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:02.807199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-40121ce2-5666-4f74-ab95-d8a7c750707b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:02.810558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 144/299] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:02.811406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe90024-4463-4de7-bc17-21106596844c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:02.883484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 156/300] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:02.884610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40121ce2-5666-4f74-ab95-d8a7c750707b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:02.921051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe90024-4463-4de7-bc17-21106596844c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:02.956253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d58658-ca6d-495d-b691-04bf3a4000fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:02.956995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d58658-ca6d-495d-b691-04bf3a4000fb None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:02.957277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d58658-ca6d-495d-b691-04bf3a4000fb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:02.957554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d58658-ca6d-495d-b691-04bf3a4000fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:02.968626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d58658-ca6d-495d-b691-04bf3a4000fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-94d58658-ca6d-495d-b691-04bf3a4000fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:02.977370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:02.978203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:02.979013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 145/301] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:02.979637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d58658-ca6d-495d-b691-04bf3a4000fb None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:02.987461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:02.991030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40121ce2-5666-4f74-ab95-d8a7c750707b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:03.024434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:03.029999 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a7cd6-0b19-4ae3-8f8e-7c5e15ee7cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:03.030927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a7cd6-0b19-4ae3-8f8e-7c5e15ee7cda None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:03.031202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a7cd6-0b19-4ae3-8f8e-7c5e15ee7cda None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:03.031513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a7cd6-0b19-4ae3-8f8e-7c5e15ee7cda None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:03.042738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a7cd6-0b19-4ae3-8f8e-7c5e15ee7cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-cd1a7cd6-0b19-4ae3-8f8e-7c5e15ee7cda None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:03.050462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:03.051329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:03.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 157/302] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:03.052794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a7cd6-0b19-4ae3-8f8e-7c5e15ee7cda None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:03.086642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d58658-ca6d-495d-b691-04bf3a4000fb None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:03.122533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031254-cd4c-4594-a256-183a8328831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:03.123340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031254-cd4c-4594-a256-183a8328831f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:03.123624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031254-cd4c-4594-a256-183a8328831f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:03.123900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031254-cd4c-4594-a256-183a8328831f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:03.125191 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031254-cd4c-4594-a256-183a8328831f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:03.126582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031254-cd4c-4594-a256-183a8328831f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:03.128243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 146/303] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:03.132887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031254-cd4c-4594-a256-183a8328831f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:03.168907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a7cd6-0b19-4ae3-8f8e-7c5e15ee7cda None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:03.204227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e3109-409f-46e6-9e6c-c64866dc3729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:03.205212 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e3109-409f-46e6-9e6c-c64866dc3729 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:03.205212 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e3109-409f-46e6-9e6c-c64866dc3729 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:03.205464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e3109-409f-46e6-9e6c-c64866dc3729 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:03.206192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e3109-409f-46e6-9e6c-c64866dc3729 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:03.206547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e3109-409f-46e6-9e6c-c64866dc3729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:03.207889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e3109-409f-46e6-9e6c-c64866dc3729 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:03.217884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 158/304] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:03.218457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e3109-409f-46e6-9e6c-c64866dc3729 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:03.244556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:03.247986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031254-cd4c-4594-a256-183a8328831f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:03.259796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:03.285188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9c33b-62b4-45e6-9ee0-35ada6b56326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:03.285928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9c33b-62b4-45e6-9ee0-35ada6b56326 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:03.286209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9c33b-62b4-45e6-9ee0-35ada6b56326 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:03.286483 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9c33b-62b4-45e6-9ee0-35ada6b56326 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:03.287241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9c33b-62b4-45e6-9ee0-35ada6b56326 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:03.287550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9c33b-62b4-45e6-9ee0-35ada6b56326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:03.289163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9c33b-62b4-45e6-9ee0-35ada6b56326 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:03.302396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 147/305] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:03.302396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9c33b-62b4-45e6-9ee0-35ada6b56326 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:03.332873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e3109-409f-46e6-9e6c-c64866dc3729 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:03.368515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540d95f-3768-45ff-be48-896ce392d679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:03.369183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540d95f-3768-45ff-be48-896ce392d679 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:03.369460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540d95f-3768-45ff-be48-896ce392d679 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:03.369743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540d95f-3768-45ff-be48-896ce392d679 None admin] PATH_INFO: `/v3/groups` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:03.370415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540d95f-3768-45ff-be48-896ce392d679 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:03.371851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540d95f-3768-45ff-be48-896ce392d679 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:03.388547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 159/306] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Jul 11 20:44:03 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:44:03.389804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540d95f-3768-45ff-be48-896ce392d679 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:03.403397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9c33b-62b4-45e6-9ee0-35ada6b56326 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:03.445442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6413b6-9f1e-4d0f-bff5-ff7a7858c2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:03.446828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6413b6-9f1e-4d0f-bff5-ff7a7858c2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:03.447358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6413b6-9f1e-4d0f-bff5-ff7a7858c2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:03.447857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6413b6-9f1e-4d0f-bff5-ff7a7858c2a0 None admin] PATH_INFO: `/v3/users/93a59339da564756914fa6fe098f209c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:03.449479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6413b6-9f1e-4d0f-bff5-ff7a7858c2a0 None admin] RBAC: Authorizing `identity:get_user(user_id=93a59339da564756914fa6fe098f209c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:03.451616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6413b6-9f1e-4d0f-bff5-ff7a7858c2a0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:03.453954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 148/307] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:44:03 2022] GET /identity/v3/users/93a59339da564756914fa6fe098f209c => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:03.459593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6413b6-9f1e-4d0f-bff5-ff7a7858c2a0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:03.504247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:03.509979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:03.530836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540d95f-3768-45ff-be48-896ce392d679 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:03.569184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f75c9-626d-431f-b4e1-83900aca8214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:03.570171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f75c9-626d-431f-b4e1-83900aca8214 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:03.570452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f75c9-626d-431f-b4e1-83900aca8214 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:03.570790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f75c9-626d-431f-b4e1-83900aca8214 None admin] PATH_INFO: `/v3/users/153099af9edb422c8d177995e2e10644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:03.580046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6413b6-9f1e-4d0f-bff5-ff7a7858c2a0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:03.589553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f75c9-626d-431f-b4e1-83900aca8214 None admin] RBAC: Authorizing `identity:get_user(user_id=153099af9edb422c8d177995e2e10644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:03.591283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f75c9-626d-431f-b4e1-83900aca8214 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:03.593351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 160/308] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:44:03 2022] GET /identity/v3/users/153099af9edb422c8d177995e2e10644 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:03.599172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f75c9-626d-431f-b4e1-83900aca8214 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:03.626252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24fa68-6812-4c52-95ae-2ecb8b928f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:03.627132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24fa68-6812-4c52-95ae-2ecb8b928f2d None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:03.627411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24fa68-6812-4c52-95ae-2ecb8b928f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:03.627684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24fa68-6812-4c52-95ae-2ecb8b928f2d None admin] PATH_INFO: `/v3/system/users/93a59339da564756914fa6fe098f209c/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:03.629848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24fa68-6812-4c52-95ae-2ecb8b928f2d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=93a59339da564756914fa6fe098f209c, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:03.632612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24fa68-6812-4c52-95ae-2ecb8b928f2d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:03.648078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 149/309] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Jul 11 20:44:03 2022] PUT /identity/v3/system/users/93a59339da564756914fa6fe098f209c/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:03.726425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f75c9-626d-431f-b4e1-83900aca8214 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:03.765907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e10018-7ba3-469d-9b59-9f4bbd62621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:03.767291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e10018-7ba3-469d-9b59-9f4bbd62621c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:03.767706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e10018-7ba3-469d-9b59-9f4bbd62621c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:03.768136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e10018-7ba3-469d-9b59-9f4bbd62621c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:03.769180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e10018-7ba3-469d-9b59-9f4bbd62621c None admin] RBAC: Adding query filter params (role.id=989e97787f6c43b5aa2603cc1b732f40, user.id=153099af9edb422c8d177995e2e10644, scope.system=all) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:03.769653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e10018-7ba3-469d-9b59-9f4bbd62621c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:03.771369 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e10018-7ba3-469d-9b59-9f4bbd62621c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:03.778365 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 161/310] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Jul 11 20:44:03 2022] GET /identity/v3/role_assignments?user.id=153099af9edb422c8d177995e2e10644&scope.system=all&role.id=989e97787f6c43b5aa2603cc1b732f40 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:04.139172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3f721-db76-4dc3-973e-9c614cd680e9 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:04.139687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3f721-db76-4dc3-973e-9c614cd680e9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:04.139790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3f721-db76-4dc3-973e-9c614cd680e9 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:04.141391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 150/311] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:04.146712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbde99-ff12-4d5d-a7d1-0cb44b391d5d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:04.146989 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbde99-ff12-4d5d-a7d1-0cb44b391d5d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:04.147266 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbde99-ff12-4d5d-a7d1-0cb44b391d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:04.153551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:04.165975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-67cbde99-ff12-4d5d-a7d1-0cb44b391d5d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:04.167158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-67cbde99-ff12-4d5d-a7d1-0cb44b391d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:04.237663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 162/312] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:04.260663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3f721-db76-4dc3-973e-9c614cd680e9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:04.366897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3f721-db76-4dc3-973e-9c614cd680e9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:04.369433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:04.405084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982c075-6184-4da1-a310-9c82b16cb1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:04.405809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982c075-6184-4da1-a310-9c82b16cb1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:04.406084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982c075-6184-4da1-a310-9c82b16cb1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:04.406408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982c075-6184-4da1-a310-9c82b16cb1c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:04.417342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982c075-6184-4da1-a310-9c82b16cb1c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-3982c075-6184-4da1-a310-9c82b16cb1c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:04.425173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:04.426040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:04.427380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 151/313] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:04.432344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbde99-ff12-4d5d-a7d1-0cb44b391d5d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:04.534798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbde99-ff12-4d5d-a7d1-0cb44b391d5d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:04.569089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196c0d2-1c2f-4159-b47d-a7e5d5b5195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:04.569814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196c0d2-1c2f-4159-b47d-a7e5d5b5195a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:04.570096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196c0d2-1c2f-4159-b47d-a7e5d5b5195a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:04.570344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196c0d2-1c2f-4159-b47d-a7e5d5b5195a None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:04.571107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196c0d2-1c2f-4159-b47d-a7e5d5b5195a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:04.571415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196c0d2-1c2f-4159-b47d-a7e5d5b5195a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:04.572822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196c0d2-1c2f-4159-b47d-a7e5d5b5195a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:04.574138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:04.582299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 163/314] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:04.588592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982c075-6184-4da1-a310-9c82b16cb1c7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:04.690356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982c075-6184-4da1-a310-9c82b16cb1c7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:04.723619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb46c4-7e3a-4a7e-9f54-6002f0816635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:04.724534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb46c4-7e3a-4a7e-9f54-6002f0816635 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:04.724812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb46c4-7e3a-4a7e-9f54-6002f0816635 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:04.725081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb46c4-7e3a-4a7e-9f54-6002f0816635 None admin] PATH_INFO: `/v3/users/69bcd53574694476a5f8a2e76839ca0a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:04.726435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb46c4-7e3a-4a7e-9f54-6002f0816635 None admin] RBAC: Authorizing `identity:get_user(user_id=69bcd53574694476a5f8a2e76839ca0a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:04.727884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb46c4-7e3a-4a7e-9f54-6002f0816635 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:04.729840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 152/315] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:44:04 2022] GET /identity/v3/users/69bcd53574694476a5f8a2e76839ca0a => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:04.735393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196c0d2-1c2f-4159-b47d-a7e5d5b5195a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:04.863805 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196c0d2-1c2f-4159-b47d-a7e5d5b5195a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:04.902964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a1118-7063-4dc7-81dd-6ff253e8c02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:04.903624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a1118-7063-4dc7-81dd-6ff253e8c02a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:04.903902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a1118-7063-4dc7-81dd-6ff253e8c02a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:04.904267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a1118-7063-4dc7-81dd-6ff253e8c02a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:04.970000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:04.977283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a1118-7063-4dc7-81dd-6ff253e8c02a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:04.979029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a1118-7063-4dc7-81dd-6ff253e8c02a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:05.075267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 164/316] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:05.081607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb46c4-7e3a-4a7e-9f54-6002f0816635 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:05.115930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:05.208772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb46c4-7e3a-4a7e-9f54-6002f0816635 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:05.249975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bea0f-1cee-430a-95b8-83527c1a7602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:05.250769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bea0f-1cee-430a-95b8-83527c1a7602 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:05.251047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bea0f-1cee-430a-95b8-83527c1a7602 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:05.251319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bea0f-1cee-430a-95b8-83527c1a7602 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:05.252534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bea0f-1cee-430a-95b8-83527c1a7602 None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:05.254086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bea0f-1cee-430a-95b8-83527c1a7602 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:05.256132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 153/317] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:05 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:05.261882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a1118-7063-4dc7-81dd-6ff253e8c02a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:05.357598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:05.363890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a1118-7063-4dc7-81dd-6ff253e8c02a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:05.398583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1d6a4-420d-4172-b6cb-9cb256554377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:05.399618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1d6a4-420d-4172-b6cb-9cb256554377 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:05.400205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1d6a4-420d-4172-b6cb-9cb256554377 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:05.400678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1d6a4-420d-4172-b6cb-9cb256554377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:05.401710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1d6a4-420d-4172-b6cb-9cb256554377 None admin] RBAC: Adding query filter params (role.id=989e97787f6c43b5aa2603cc1b732f40, scope.project.id=8c32c58c2df149f4842efa86dea7e18f, user.id=69bcd53574694476a5f8a2e76839ca0a) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:05.402206 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1d6a4-420d-4172-b6cb-9cb256554377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:05.403873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1d6a4-420d-4172-b6cb-9cb256554377 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:05.410952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 165/318] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:05 2022] GET /identity/v3/role_assignments?user.id=69bcd53574694476a5f8a2e76839ca0a&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=989e97787f6c43b5aa2603cc1b732f40 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:05.555136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a77a04-176c-4fdd-8563-389eb1e6e7d2 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:05.555486 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a77a04-176c-4fdd-8563-389eb1e6e7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:05.555652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a77a04-176c-4fdd-8563-389eb1e6e7d2 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:05.557391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 154/319] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:05.563169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a022d8-c0a7-4dec-b353-801409ec870d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:05.563444 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a022d8-c0a7-4dec-b353-801409ec870d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:05.563722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a022d8-c0a7-4dec-b353-801409ec870d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:05.582520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-09a022d8-c0a7-4dec-b353-801409ec870d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:05.583722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-09a022d8-c0a7-4dec-b353-801409ec870d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:05.652806 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 166/320] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:05.675346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a77a04-176c-4fdd-8563-389eb1e6e7d2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:05.784336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:05.789078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a77a04-176c-4fdd-8563-389eb1e6e7d2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:05.809675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d83e39-e622-491e-ba6d-29569f49d727 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:05.809962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d83e39-e622-491e-ba6d-29569f49d727 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:05.810685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d83e39-e622-491e-ba6d-29569f49d727 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:05.812230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 167/321] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:05.817259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a16f0-c8e4-4040-89e3-8ce3a0bd34f2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:05.817540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a16f0-c8e4-4040-89e3-8ce3a0bd34f2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:05.817811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a16f0-c8e4-4040-89e3-8ce3a0bd34f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:05.827924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e52dd9-f11e-4c46-9e82-64fb4228f91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:05.829090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e52dd9-f11e-4c46-9e82-64fb4228f91e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:05.829550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e52dd9-f11e-4c46-9e82-64fb4228f91e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:05.830000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e52dd9-f11e-4c46-9e82-64fb4228f91e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:05.831391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e52dd9-f11e-4c46-9e82-64fb4228f91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:05.833084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e52dd9-f11e-4c46-9e82-64fb4228f91e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:05.834947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 155/322] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:05.835506 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-925a16f0-c8e4-4040-89e3-8ce3a0bd34f2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:05.836783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-925a16f0-c8e4-4040-89e3-8ce3a0bd34f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:05.840150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e52dd9-f11e-4c46-9e82-64fb4228f91e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:05.905466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 168/323] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:05.906783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea0f99-1e54-4368-8634-da09e73319c4 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:05.907068 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea0f99-1e54-4368-8634-da09e73319c4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:05.907337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea0f99-1e54-4368-8634-da09e73319c4 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:05.912148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 169/324] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:05.916903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808b23d-932c-4894-8219-5d4279e589d6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:05.917564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808b23d-932c-4894-8219-5d4279e589d6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:05.917843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808b23d-932c-4894-8219-5d4279e589d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:05.935467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1808b23d-932c-4894-8219-5d4279e589d6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:05.936749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1808b23d-932c-4894-8219-5d4279e589d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:05.939936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e52dd9-f11e-4c46-9e82-64fb4228f91e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:05.951027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:05.972314 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4fb90-c636-4602-a098-5d23074794c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:05.972984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4fb90-c636-4602-a098-5d23074794c5 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:05.973261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4fb90-c636-4602-a098-5d23074794c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:05.973531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4fb90-c636-4602-a098-5d23074794c5 None admin] PATH_INFO: `/v3/groups` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:05.974210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4fb90-c636-4602-a098-5d23074794c5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:05.975716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4fb90-c636-4602-a098-5d23074794c5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:05.991226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 156/325] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Jul 11 20:44:05 2022] POST /identity/v3/groups => generated 226 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:44:05.997183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4fb90-c636-4602-a098-5d23074794c5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:06.016143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 170/326] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:06.030491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:06.031658 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808b23d-932c-4894-8219-5d4279e589d6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:06.116083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4fb90-c636-4602-a098-5d23074794c5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:06.143940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808b23d-932c-4894-8219-5d4279e589d6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:06.153540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3baaf4-0fa0-43fc-bc46-a7b98d1eed8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:06.154391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3baaf4-0fa0-43fc-bc46-a7b98d1eed8e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:06.154677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3baaf4-0fa0-43fc-bc46-a7b98d1eed8e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:06.154952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3baaf4-0fa0-43fc-bc46-a7b98d1eed8e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:06.166680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3baaf4-0fa0-43fc-bc46-a7b98d1eed8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-2f3baaf4-0fa0-43fc-bc46-a7b98d1eed8e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:06.174481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:06.175462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:06.176354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 157/327] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:06.182311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3baaf4-0fa0-43fc-bc46-a7b98d1eed8e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:06.185023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a71a63-4c54-41d8-a0e9-c79c9edce2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:06.186083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71a63-4c54-41d8-a0e9-c79c9edce2de None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:06.186185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71a63-4c54-41d8-a0e9-c79c9edce2de None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:06.186417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71a63-4c54-41d8-a0e9-c79c9edce2de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:06.198616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a71a63-4c54-41d8-a0e9-c79c9edce2de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-67a71a63-4c54-41d8-a0e9-c79c9edce2de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:06.206567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:06.207427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:06.208259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 171/328] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:06.214028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a71a63-4c54-41d8-a0e9-c79c9edce2de None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:06.227819 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:06.292675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:06.309604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3baaf4-0fa0-43fc-bc46-a7b98d1eed8e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:06.348890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a71a63-4c54-41d8-a0e9-c79c9edce2de None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:06.350759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac42fd8-7f63-43ce-87da-ac71d033d376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:06.351988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac42fd8-7f63-43ce-87da-ac71d033d376 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:06.351988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac42fd8-7f63-43ce-87da-ac71d033d376 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:06.351988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac42fd8-7f63-43ce-87da-ac71d033d376 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:06.352765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac42fd8-7f63-43ce-87da-ac71d033d376 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:06.353070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac42fd8-7f63-43ce-87da-ac71d033d376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:06.354511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac42fd8-7f63-43ce-87da-ac71d033d376 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:06.365127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 158/329] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:06.370665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac42fd8-7f63-43ce-87da-ac71d033d376 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:06.387946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9cbff-66c5-4412-9d64-92cdf643659f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:06.388696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9cbff-66c5-4412-9d64-92cdf643659f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:06.388977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9cbff-66c5-4412-9d64-92cdf643659f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:06.389302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9cbff-66c5-4412-9d64-92cdf643659f None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:06.390062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9cbff-66c5-4412-9d64-92cdf643659f None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:06.390374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9cbff-66c5-4412-9d64-92cdf643659f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:06.391854 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9cbff-66c5-4412-9d64-92cdf643659f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:06.405575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 172/330] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:06.410816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9cbff-66c5-4412-9d64-92cdf643659f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:06.500761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:06.510895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac42fd8-7f63-43ce-87da-ac71d033d376 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:06.539985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9cbff-66c5-4412-9d64-92cdf643659f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:06.565849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5099af-cfd6-4e26-9e33-68a2e68feb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:06.567383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5099af-cfd6-4e26-9e33-68a2e68feb67 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:06.567876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5099af-cfd6-4e26-9e33-68a2e68feb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:06.568491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5099af-cfd6-4e26-9e33-68a2e68feb67 None admin] PATH_INFO: `/v3/users/153099af9edb422c8d177995e2e10644` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:06.570722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5099af-cfd6-4e26-9e33-68a2e68feb67 None admin] RBAC: Authorizing `identity:get_user(user_id=153099af9edb422c8d177995e2e10644)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:06.572937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5099af-cfd6-4e26-9e33-68a2e68feb67 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:06.580323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 159/331] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:44:06 2022] GET /identity/v3/users/153099af9edb422c8d177995e2e10644 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:06.582328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5099af-cfd6-4e26-9e33-68a2e68feb67 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:06.586243 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1f03d-e7ef-496a-9978-90b52e0b3db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:06.587146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1f03d-e7ef-496a-9978-90b52e0b3db4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:06.587428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1f03d-e7ef-496a-9978-90b52e0b3db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:06.587701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1f03d-e7ef-496a-9978-90b52e0b3db4 None admin] PATH_INFO: `/v3/users/93a59339da564756914fa6fe098f209c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:06.589536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1f03d-e7ef-496a-9978-90b52e0b3db4 None admin] RBAC: Authorizing `identity:get_user(user_id=93a59339da564756914fa6fe098f209c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:06.591008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1f03d-e7ef-496a-9978-90b52e0b3db4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:06.592858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 173/332] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:44:06 2022] GET /identity/v3/users/93a59339da564756914fa6fe098f209c => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:06.600165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1f03d-e7ef-496a-9978-90b52e0b3db4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:06.611749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:06.718296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5099af-cfd6-4e26-9e33-68a2e68feb67 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:06.735526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1f03d-e7ef-496a-9978-90b52e0b3db4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:06.756174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac1fc5-ed62-403e-ac6b-892179b3b7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:06.757122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac1fc5-ed62-403e-ac6b-892179b3b7ab None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:06.757405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac1fc5-ed62-403e-ac6b-892179b3b7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:06.757709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac1fc5-ed62-403e-ac6b-892179b3b7ab None admin] PATH_INFO: `/v3/system/users/153099af9edb422c8d177995e2e10644/roles/989e97787f6c43b5aa2603cc1b732f40` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:06.760295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac1fc5-ed62-403e-ac6b-892179b3b7ab None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=153099af9edb422c8d177995e2e10644, role_id=989e97787f6c43b5aa2603cc1b732f40)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:06.761684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac1fc5-ed62-403e-ac6b-892179b3b7ab None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:06.775154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 160/333] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Jul 11 20:44:06 2022] PUT /identity/v3/system/users/153099af9edb422c8d177995e2e10644/roles/989e97787f6c43b5aa2603cc1b732f40 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:06.780433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90040eac-5c22-4a92-a227-1df4bc35c02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:06.781632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90040eac-5c22-4a92-a227-1df4bc35c02b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:06.782176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90040eac-5c22-4a92-a227-1df4bc35c02b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:06.782709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90040eac-5c22-4a92-a227-1df4bc35c02b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:06.783905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90040eac-5c22-4a92-a227-1df4bc35c02b None admin] RBAC: Adding query filter params (role.id=1403cb7e4e264c68b63e467a2e433699, user.id=93a59339da564756914fa6fe098f209c, scope.system=all) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:06.784436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90040eac-5c22-4a92-a227-1df4bc35c02b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:06.786700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90040eac-5c22-4a92-a227-1df4bc35c02b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:06.795338 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:06.807211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 174/334] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Jul 11 20:44:06 2022] GET /identity/v3/role_assignments?user.id=93a59339da564756914fa6fe098f209c&scope.system=all&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:08.292200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64a85d-aab8-4a6d-963c-12fb6f2bed75 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:08.292720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64a85d-aab8-4a6d-963c-12fb6f2bed75 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:08.292794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64a85d-aab8-4a6d-963c-12fb6f2bed75 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:08.294413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 161/335] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:08.306996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee04a5-5686-43e1-bec4-f48bb32046ec None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:08.307485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee04a5-5686-43e1-bec4-f48bb32046ec None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:08.307559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee04a5-5686-43e1-bec4-f48bb32046ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:08.326548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7cee04a5-5686-43e1-bec4-f48bb32046ec None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:08.327747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7cee04a5-5686-43e1-bec4-f48bb32046ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:08.409484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 175/336] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:08.432244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64a85d-aab8-4a6d-963c-12fb6f2bed75 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:08.542068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:08.564819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64a85d-aab8-4a6d-963c-12fb6f2bed75 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:08.610684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0713d-c82d-4dd3-8e44-a8990573a9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:08.611889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0713d-c82d-4dd3-8e44-a8990573a9fa None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:08.612385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0713d-c82d-4dd3-8e44-a8990573a9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:08.612847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0713d-c82d-4dd3-8e44-a8990573a9fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:08.627308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0713d-c82d-4dd3-8e44-a8990573a9fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-66c0713d-c82d-4dd3-8e44-a8990573a9fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:08.635893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:08.636851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:08.637620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:08.640036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 162/337] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:08.646106 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee04a5-5686-43e1-bec4-f48bb32046ec None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:08.771458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee04a5-5686-43e1-bec4-f48bb32046ec None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:08.803819 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:08.814646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1aaf77-91b9-4e10-897a-e38bfd9b0d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:08.815383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1aaf77-91b9-4e10-897a-e38bfd9b0d97 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:08.816016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1aaf77-91b9-4e10-897a-e38bfd9b0d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:08.817704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1aaf77-91b9-4e10-897a-e38bfd9b0d97 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:08.818481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1aaf77-91b9-4e10-897a-e38bfd9b0d97 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:08.818788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1aaf77-91b9-4e10-897a-e38bfd9b0d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:08.820534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1aaf77-91b9-4e10-897a-e38bfd9b0d97 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:08.834559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 176/338] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:08.840625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0713d-c82d-4dd3-8e44-a8990573a9fa None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:08.990944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0713d-c82d-4dd3-8e44-a8990573a9fa None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:09.027114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b4fdb-ddce-4a7f-b0e3-d938a7473764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:09.028011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b4fdb-ddce-4a7f-b0e3-d938a7473764 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.028448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b4fdb-ddce-4a7f-b0e3-d938a7473764 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.028857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b4fdb-ddce-4a7f-b0e3-d938a7473764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.029020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208046-ac6c-4222-97e2-094c0537737c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.029312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208046-ac6c-4222-97e2-094c0537737c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.029589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208046-ac6c-4222-97e2-094c0537737c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.031205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 177/339] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:09.037118 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f59018-0895-4912-b8df-61e05e7b96ea None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.037402 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f59018-0895-4912-b8df-61e05e7b96ea None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.037683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f59018-0895-4912-b8df-61e05e7b96ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.049911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:09.073239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d1f59018-0895-4912-b8df-61e05e7b96ea None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:09.076490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d1f59018-0895-4912-b8df-61e05e7b96ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:09.136099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b4fdb-ddce-4a7f-b0e3-d938a7473764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:09.137842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b4fdb-ddce-4a7f-b0e3-d938a7473764 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:09.179568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 178/340] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:09.181257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be4a5e-3242-4c8b-8352-21f47c3b056b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.181611 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be4a5e-3242-4c8b-8352-21f47c3b056b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.181950 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be4a5e-3242-4c8b-8352-21f47c3b056b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.184223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 179/341] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:09.189319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8db98-7312-412d-87b0-d1b8f00f8e99 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.189790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8db98-7312-412d-87b0-d1b8f00f8e99 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.190219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8db98-7312-412d-87b0-d1b8f00f8e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.212292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-49b8db98-7312-412d-87b0-d1b8f00f8e99 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:09.213743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-49b8db98-7312-412d-87b0-d1b8f00f8e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:09.265441 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 163/342] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:09.266349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b4fdb-ddce-4a7f-b0e3-d938a7473764 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:09.291845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 180/343] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:09.293108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8db98-7312-412d-87b0-d1b8f00f8e99 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:09.318465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:09.348083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:09.379470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b4fdb-ddce-4a7f-b0e3-d938a7473764 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:09.402113 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8db98-7312-412d-87b0-d1b8f00f8e99 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:09.418634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103d89d-f7d3-4f47-b158-78d5508c73bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:09.419468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103d89d-f7d3-4f47-b158-78d5508c73bf None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.419745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103d89d-f7d3-4f47-b158-78d5508c73bf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.420124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103d89d-f7d3-4f47-b158-78d5508c73bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.431813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103d89d-f7d3-4f47-b158-78d5508c73bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9103d89d-f7d3-4f47-b158-78d5508c73bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:09.441569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:09.442460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:09.443662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 164/344] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:44:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:09.444880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103d89d-f7d3-4f47-b158-78d5508c73bf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:09.445373 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d857e-e3af-41fc-91e7-37668005a481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:09.446498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d857e-e3af-41fc-91e7-37668005a481 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.446965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d857e-e3af-41fc-91e7-37668005a481 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.447508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d857e-e3af-41fc-91e7-37668005a481 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.449559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d857e-e3af-41fc-91e7-37668005a481 None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:09.451685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d857e-e3af-41fc-91e7-37668005a481 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:09.454022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 181/345] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:09 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:09.455018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d857e-e3af-41fc-91e7-37668005a481 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:09.556918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103d89d-f7d3-4f47-b158-78d5508c73bf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:09.558549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:09.567444 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d857e-e3af-41fc-91e7-37668005a481 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:09.587183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:09.597131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcd160-2b3f-49ce-b113-339beff3be7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:09.597311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcd160-2b3f-49ce-b113-339beff3be7f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.597574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcd160-2b3f-49ce-b113-339beff3be7f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.597849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcd160-2b3f-49ce-b113-339beff3be7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.607972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a31af8-9376-4b56-8b15-9e410e5817e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:09.608953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a31af8-9376-4b56-8b15-9e410e5817e3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.609449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a31af8-9376-4b56-8b15-9e410e5817e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.609724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcd160-2b3f-49ce-b113-339beff3be7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:09.610132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a31af8-9376-4b56-8b15-9e410e5817e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.611078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a31af8-9376-4b56-8b15-9e410e5817e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:09.611571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a31af8-9376-4b56-8b15-9e410e5817e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:09.613179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a31af8-9376-4b56-8b15-9e410e5817e3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e7fcd160-2b3f-49ce-b113-339beff3be7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:09.617480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:09.618372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:09.619668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 165/346] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:09.620555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcd160-2b3f-49ce-b113-339beff3be7f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:09.625897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 182/347] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:09.627086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a31af8-9376-4b56-8b15-9e410e5817e3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:09.738599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a31af8-9376-4b56-8b15-9e410e5817e3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:09.775714 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcd160-2b3f-49ce-b113-339beff3be7f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:09.779719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc2b0a-ed3f-4e45-ab61-022aa3db6086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:09.780654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc2b0a-ed3f-4e45-ab61-022aa3db6086 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.781130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc2b0a-ed3f-4e45-ab61-022aa3db6086 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.781593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc2b0a-ed3f-4e45-ab61-022aa3db6086 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.782547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc2b0a-ed3f-4e45-ab61-022aa3db6086 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:09.783060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc2b0a-ed3f-4e45-ab61-022aa3db6086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:09.784674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc2b0a-ed3f-4e45-ab61-022aa3db6086 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:09.802304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 183/348] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:09.804738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc2b0a-ed3f-4e45-ab61-022aa3db6086 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:09.845728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0aa71-a540-44d2-babc-c295d8a13f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:09.847292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0aa71-a540-44d2-babc-c295d8a13f66 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:09.848038 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0aa71-a540-44d2-babc-c295d8a13f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:09.848546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0aa71-a540-44d2-babc-c295d8a13f66 None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:09.857897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0aa71-a540-44d2-babc-c295d8a13f66 None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:09.860000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0aa71-a540-44d2-babc-c295d8a13f66 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:09.863112 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 166/349] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:09 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:09.868208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0aa71-a540-44d2-babc-c295d8a13f66 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:09.899898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:09.925934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:10.010628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc2b0a-ed3f-4e45-ab61-022aa3db6086 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:10.051458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0aa71-a540-44d2-babc-c295d8a13f66 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:10.074976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c8c43-d037-4632-b059-70efbca72b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:10.076372 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c8c43-d037-4632-b059-70efbca72b4d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:10.076848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c8c43-d037-4632-b059-70efbca72b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:10.077333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c8c43-d037-4632-b059-70efbca72b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:10.117023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27e8be-25f2-4114-a877-3cc894f7ee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:10.119060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27e8be-25f2-4114-a877-3cc894f7ee7a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:10.119536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27e8be-25f2-4114-a877-3cc894f7ee7a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:10.120187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27e8be-25f2-4114-a877-3cc894f7ee7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:10.204727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c8c43-d037-4632-b059-70efbca72b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:10.206592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c8c43-d037-4632-b059-70efbca72b4d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:10.241074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27e8be-25f2-4114-a877-3cc894f7ee7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:10.242806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27e8be-25f2-4114-a877-3cc894f7ee7a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:10.270418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:10.281976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:10.341683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 184/350] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:10.342567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c8c43-d037-4632-b059-70efbca72b4d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:10.385461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 167/351] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:10.386737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8296f2d-e6f6-47c8-ac33-8847c73275f0 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:10.387020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8296f2d-e6f6-47c8-ac33-8847c73275f0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:10.387292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8296f2d-e6f6-47c8-ac33-8847c73275f0 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:10.388831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 168/352] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:10.390057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8296f2d-e6f6-47c8-ac33-8847c73275f0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:10.484315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c8c43-d037-4632-b059-70efbca72b4d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:10.523323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac3976-4941-453d-b32c-d927a228d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:10.523678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3976-4941-453d-b32c-d927a228d4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:10.523949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3976-4941-453d-b32c-d927a228d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:10.524310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3976-4941-453d-b32c-d927a228d4f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:10.525261 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac3976-4941-453d-b32c-d927a228d4f2 None admin] RBAC: Adding query filter params (group.id=1e773e646bd7466eb00fe330aab338f2, role.id=1403cb7e4e264c68b63e467a2e433699, scope.project.id=3ad2772943334037bb33953f59d816e3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:10.525572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac3976-4941-453d-b32c-d927a228d4f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:10.527011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac3976-4941-453d-b32c-d927a228d4f2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:10.533461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 185/353] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jul 11 20:44:10 2022] GET /identity/v3/role_assignments?group.id=1e773e646bd7466eb00fe330aab338f2&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:10.534707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7851021-c84d-4346-9489-b6e803b48214 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:10.534987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7851021-c84d-4346-9489-b6e803b48214 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:10.535259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7851021-c84d-4346-9489-b6e803b48214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:10.549412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8296f2d-e6f6-47c8-ac33-8847c73275f0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:10.555545 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:10.566154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c7851021-c84d-4346-9489-b6e803b48214 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:10.568463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c7851021-c84d-4346-9489-b6e803b48214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:10.586898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:10.598884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c8374-845d-4c5b-aee8-a0480b45de8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:10.600772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c8374-845d-4c5b-aee8-a0480b45de8c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:10.601836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c8374-845d-4c5b-aee8-a0480b45de8c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:10.602522 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c8374-845d-4c5b-aee8-a0480b45de8c None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:10.604788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c8374-845d-4c5b-aee8-a0480b45de8c None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:10.606482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c8374-845d-4c5b-aee8-a0480b45de8c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:10.608893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 169/354] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:10 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:10.610046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c8374-845d-4c5b-aee8-a0480b45de8c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:10.647124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 186/355] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:10.647124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7851021-c84d-4346-9489-b6e803b48214 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:10.755298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7851021-c84d-4346-9489-b6e803b48214 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:10.774930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c8374-845d-4c5b-aee8-a0480b45de8c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:10.793563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ede2-7de2-4bf4-9711-a21f3a3ccadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:10.794506 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ede2-7de2-4bf4-9711-a21f3a3ccadf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:10.794785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ede2-7de2-4bf4-9711-a21f3a3ccadf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:10.795051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ede2-7de2-4bf4-9711-a21f3a3ccadf None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:10.796460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ede2-7de2-4bf4-9711-a21f3a3ccadf None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:10.798114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ede2-7de2-4bf4-9711-a21f3a3ccadf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:10.799835 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 187/356] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:10 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:10.800660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ede2-7de2-4bf4-9711-a21f3a3ccadf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:10.804684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:10.844728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44c87f-26ab-4932-a52a-49725a4e244a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:10.845643 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44c87f-26ab-4932-a52a-49725a4e244a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:10.845914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44c87f-26ab-4932-a52a-49725a4e244a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:10.846193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44c87f-26ab-4932-a52a-49725a4e244a None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:10.847517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44c87f-26ab-4932-a52a-49725a4e244a None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:10.849171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44c87f-26ab-4932-a52a-49725a4e244a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:10.851359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 170/357] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:10 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:10.856520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44c87f-26ab-4932-a52a-49725a4e244a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:10.913697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ede2-7de2-4bf4-9711-a21f3a3ccadf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:10.926305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:10.952258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96224f5-e874-4da9-9a32-d4a9afc6b81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:10.953076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96224f5-e874-4da9-9a32-d4a9afc6b81f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:10.953356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96224f5-e874-4da9-9a32-d4a9afc6b81f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:10.953626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96224f5-e874-4da9-9a32-d4a9afc6b81f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:10.964934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96224f5-e874-4da9-9a32-d4a9afc6b81f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f96224f5-e874-4da9-9a32-d4a9afc6b81f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:10.972643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:10.973708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 20:44:10.974636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:10.975805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44c87f-26ab-4932-a52a-49725a4e244a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:10.977156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 188/358] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:10.978269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c6611-6e4f-43b5-a14a-dedb6ee10226 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:10.978559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c6611-6e4f-43b5-a14a-dedb6ee10226 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:10.978831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c6611-6e4f-43b5-a14a-dedb6ee10226 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:10.980124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 189/359] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:10.981422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c6611-6e4f-43b5-a14a-dedb6ee10226 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:11.019369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf403add-e661-4221-b900-b8a7bc32e5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:11.020595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf403add-e661-4221-b900-b8a7bc32e5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.021032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf403add-e661-4221-b900-b8a7bc32e5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.021437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf403add-e661-4221-b900-b8a7bc32e5c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.022454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf403add-e661-4221-b900-b8a7bc32e5c3 None admin] RBAC: Adding query filter params (group.id=1e773e646bd7466eb00fe330aab338f2, role.id=e2c9006dc77f473f8e1064f285400a98, scope.project.id=3ad2772943334037bb33953f59d816e3) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:11.022903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf403add-e661-4221-b900-b8a7bc32e5c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:11.025670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf403add-e661-4221-b900-b8a7bc32e5c3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:11.032809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 171/360] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jul 11 20:44:10 2022] GET /identity/v3/role_assignments?group.id=1e773e646bd7466eb00fe330aab338f2&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=e2c9006dc77f473f8e1064f285400a98 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:11.034242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacf1f4-8a05-4420-89ee-55c36809d0e0 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.034528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacf1f4-8a05-4420-89ee-55c36809d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.034802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacf1f4-8a05-4420-89ee-55c36809d0e0 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.036528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 172/361] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:11.037210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacf1f4-8a05-4420-89ee-55c36809d0e0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:11.060185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:11.111711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c6611-6e4f-43b5-a14a-dedb6ee10226 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:11.156667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491b4ee-953e-4f35-b267-5b2d9ed84fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:11.157538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491b4ee-953e-4f35-b267-5b2d9ed84fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.157886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491b4ee-953e-4f35-b267-5b2d9ed84fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.158223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491b4ee-953e-4f35-b267-5b2d9ed84fc1 None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.160563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491b4ee-953e-4f35-b267-5b2d9ed84fc1 None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:11.162269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491b4ee-953e-4f35-b267-5b2d9ed84fc1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:11.165292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 190/362] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:10 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:11.170589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75656e4e-5b3e-4ece-a050-3059266e79b1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.170589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75656e4e-5b3e-4ece-a050-3059266e79b1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.170589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75656e4e-5b3e-4ece-a050-3059266e79b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.178537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacf1f4-8a05-4420-89ee-55c36809d0e0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:11.191161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-75656e4e-5b3e-4ece-a050-3059266e79b1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:11.192408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-75656e4e-5b3e-4ece-a050-3059266e79b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:11.222624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:11.235796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0359d2-a766-459b-be87-4510c8b824f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:11.236951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0359d2-a766-459b-be87-4510c8b824f1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.237499 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0359d2-a766-459b-be87-4510c8b824f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.237774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0359d2-a766-459b-be87-4510c8b824f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.239534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0359d2-a766-459b-be87-4510c8b824f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:11.239843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0359d2-a766-459b-be87-4510c8b824f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:11.241873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0359d2-a766-459b-be87-4510c8b824f1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:11.254121 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 173/363] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:11.254715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fbbf6-202e-404a-b86e-5135b81052e4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.255153 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fbbf6-202e-404a-b86e-5135b81052e4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.255433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fbbf6-202e-404a-b86e-5135b81052e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.273930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 191/364] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:11.274434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6e4fbbf6-202e-404a-b86e-5135b81052e4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:11.275813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6e4fbbf6-202e-404a-b86e-5135b81052e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:11.276265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75656e4e-5b3e-4ece-a050-3059266e79b1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:11.352967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 174/365] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:11.353390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:11.353897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fbbf6-202e-404a-b86e-5135b81052e4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:11.401256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75656e4e-5b3e-4ece-a050-3059266e79b1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:11.445840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7a7d8-b34f-46d7-9e64-7defaaa185dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:11.446438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7a7d8-b34f-46d7-9e64-7defaaa185dd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.446722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7a7d8-b34f-46d7-9e64-7defaaa185dd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.447000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7a7d8-b34f-46d7-9e64-7defaaa185dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.447847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7a7d8-b34f-46d7-9e64-7defaaa185dd None admin] RBAC: Adding query filter params (group.id=1e773e646bd7466eb00fe330aab338f2, role.id=1403cb7e4e264c68b63e467a2e433699, scope.project.id=8c32c58c2df149f4842efa86dea7e18f) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:11.448193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7a7d8-b34f-46d7-9e64-7defaaa185dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:11.450609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7a7d8-b34f-46d7-9e64-7defaaa185dd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:11.459178 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 192/366] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jul 11 20:44:11 2022] GET /identity/v3/role_assignments?group.id=1e773e646bd7466eb00fe330aab338f2&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:11.460539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7a7d8-b34f-46d7-9e64-7defaaa185dd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:11.484301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fbbf6-202e-404a-b86e-5135b81052e4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:11.510194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:11.552149 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512f43d-13d9-4354-aa52-1be1a765e00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:11.553136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512f43d-13d9-4354-aa52-1be1a765e00c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.553417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512f43d-13d9-4354-aa52-1be1a765e00c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.553807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512f43d-13d9-4354-aa52-1be1a765e00c None admin] PATH_INFO: `/v3/users/153099af9edb422c8d177995e2e10644` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.555477 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512f43d-13d9-4354-aa52-1be1a765e00c None admin] RBAC: Authorizing `identity:get_user(user_id=153099af9edb422c8d177995e2e10644)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:11.557010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512f43d-13d9-4354-aa52-1be1a765e00c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:11.562208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 175/367] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:44:11 2022] GET /identity/v3/users/153099af9edb422c8d177995e2e10644 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:11.565339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512f43d-13d9-4354-aa52-1be1a765e00c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:11.655530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7a7d8-b34f-46d7-9e64-7defaaa185dd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:11.689393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:11.728953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75d7cd-cdcb-4008-9ead-3fab2cf4ea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:11.729716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75d7cd-cdcb-4008-9ead-3fab2cf4ea9d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.730001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75d7cd-cdcb-4008-9ead-3fab2cf4ea9d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.730278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75d7cd-cdcb-4008-9ead-3fab2cf4ea9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.747824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75d7cd-cdcb-4008-9ead-3fab2cf4ea9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:11.752039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512f43d-13d9-4354-aa52-1be1a765e00c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1d75d7cd-cdcb-4008-9ead-3fab2cf4ea9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:11.755998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:11.757183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:11.758190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:11.763530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 193/368] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:44:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:11.764447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75d7cd-cdcb-4008-9ead-3fab2cf4ea9d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:11.802310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c2191-d4af-45ec-8aa2-ec9c32af6775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:11.803522 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c2191-d4af-45ec-8aa2-ec9c32af6775 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.804412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c2191-d4af-45ec-8aa2-ec9c32af6775 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.804907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c2191-d4af-45ec-8aa2-ec9c32af6775 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.816775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c2191-d4af-45ec-8aa2-ec9c32af6775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-393c2191-d4af-45ec-8aa2-ec9c32af6775 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:11.825675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:11.826542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:11.827317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:11.830483 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 176/369] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:11.831782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c2191-d4af-45ec-8aa2-ec9c32af6775 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:11.891665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:11.946463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75d7cd-cdcb-4008-9ead-3fab2cf4ea9d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:11.981978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c2191-d4af-45ec-8aa2-ec9c32af6775 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:11.993392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf3c56-f3f9-4440-8ad5-39a35eee7470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:11.994076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf3c56-f3f9-4440-8ad5-39a35eee7470 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:11.994359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf3c56-f3f9-4440-8ad5-39a35eee7470 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:11.994636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf3c56-f3f9-4440-8ad5-39a35eee7470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:11.995595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf3c56-f3f9-4440-8ad5-39a35eee7470 None admin] RBAC: Adding query filter params (role.id=989e97787f6c43b5aa2603cc1b732f40, user.id=153099af9edb422c8d177995e2e10644, scope.system=all) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:11.995908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf3c56-f3f9-4440-8ad5-39a35eee7470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:11.999263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf3c56-f3f9-4440-8ad5-39a35eee7470 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:12.015029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 194/370] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Jul 11 20:44:11 2022] GET /identity/v3/role_assignments?user.id=153099af9edb422c8d177995e2e10644&scope.system=all&role.id=989e97787f6c43b5aa2603cc1b732f40 => generated 510 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:12.017250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf3c56-f3f9-4440-8ad5-39a35eee7470 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:12.034434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4b58-b0b5-45cd-af2e-d8e85a92dfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:12.035555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4b58-b0b5-45cd-af2e-d8e85a92dfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:12.036074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4b58-b0b5-45cd-af2e-d8e85a92dfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:12.036612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4b58-b0b5-45cd-af2e-d8e85a92dfe6 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:12.037706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d4b58-b0b5-45cd-af2e-d8e85a92dfe6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:12.038289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d4b58-b0b5-45cd-af2e-d8e85a92dfe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:12.039938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d4b58-b0b5-45cd-af2e-d8e85a92dfe6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:12.049775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 177/371] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:12.055647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4b58-b0b5-45cd-af2e-d8e85a92dfe6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:12.110216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:12.178751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf3c56-f3f9-4440-8ad5-39a35eee7470 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:12.227743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55306137-dc76-4609-853d-01201e5398bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:12.230411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55306137-dc76-4609-853d-01201e5398bf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:12.231141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55306137-dc76-4609-853d-01201e5398bf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:12.231820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55306137-dc76-4609-853d-01201e5398bf None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:12.233787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55306137-dc76-4609-853d-01201e5398bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:12.234502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55306137-dc76-4609-853d-01201e5398bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:12.237900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55306137-dc76-4609-853d-01201e5398bf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:12.260216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 195/372] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:12.266314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55306137-dc76-4609-853d-01201e5398bf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:12.278304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4b58-b0b5-45cd-af2e-d8e85a92dfe6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:12.294281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:12.385404 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afae821-f7d2-4b03-b41c-64101720d002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:12.386113 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afae821-f7d2-4b03-b41c-64101720d002 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:12.386394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afae821-f7d2-4b03-b41c-64101720d002 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:12.386664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afae821-f7d2-4b03-b41c-64101720d002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:12.407279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55306137-dc76-4609-853d-01201e5398bf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:12.422218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:12.463847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaee059-ea1d-4152-8263-c83cfb9c3b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:12.464934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaee059-ea1d-4152-8263-c83cfb9c3b80 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:12.465572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaee059-ea1d-4152-8263-c83cfb9c3b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:12.468025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaee059-ea1d-4152-8263-c83cfb9c3b80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:12.553224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afae821-f7d2-4b03-b41c-64101720d002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:12.555403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afae821-f7d2-4b03-b41c-64101720d002 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:12.598106 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaee059-ea1d-4152-8263-c83cfb9c3b80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:12.600006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaee059-ea1d-4152-8263-c83cfb9c3b80 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:12.708315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 196/373] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:12.713733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaee059-ea1d-4152-8263-c83cfb9c3b80 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:12.726606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:12.729797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 178/374] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:12.736136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afae821-f7d2-4b03-b41c-64101720d002 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:12.747729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:12.887128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaee059-ea1d-4152-8263-c83cfb9c3b80 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:12.898778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afae821-f7d2-4b03-b41c-64101720d002 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:12.953267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c8f7-5cc5-4349-881f-f800c64f3ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:12.954141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c8f7-5cc5-4349-881f-f800c64f3ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:12.954419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c8f7-5cc5-4349-881f-f800c64f3ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:12.954690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c8f7-5cc5-4349-881f-f800c64f3ca7 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:12.956890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4c8f7-5cc5-4349-881f-f800c64f3ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:12.958349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4c8f7-5cc5-4349-881f-f800c64f3ca7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:12.960500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 179/375] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:12 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:12.967354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c8f7-5cc5-4349-881f-f800c64f3ca7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:12.970698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b3ed2-78d7-414e-add0-82a809e749b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:12.971865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3ed2-78d7-414e-add0-82a809e749b1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:12.972379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3ed2-78d7-414e-add0-82a809e749b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:12.972808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3ed2-78d7-414e-add0-82a809e749b1 None admin] PATH_INFO: `/v3/projects/27db080494394cc5b886636221f8edb9` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:12.974213 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b3ed2-78d7-414e-add0-82a809e749b1 None admin] RBAC: Authorizing `identity:get_project(project_id=27db080494394cc5b886636221f8edb9)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:12.975864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b3ed2-78d7-414e-add0-82a809e749b1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:12.978207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 197/376] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:12 2022] GET /identity/v3/projects/27db080494394cc5b886636221f8edb9 => generated 344 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:12.983811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b3ed2-78d7-414e-add0-82a809e749b1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:13.093663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:13.103368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:13.140275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b3ed2-78d7-414e-add0-82a809e749b1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:13.144471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c8f7-5cc5-4349-881f-f800c64f3ca7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:13.188745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8a3f2-896e-4afe-8b8d-15412d8b024d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:13.189952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8a3f2-896e-4afe-8b8d-15412d8b024d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:13.190373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8a3f2-896e-4afe-8b8d-15412d8b024d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:13.190784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8a3f2-896e-4afe-8b8d-15412d8b024d None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:13.194265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8a3f2-896e-4afe-8b8d-15412d8b024d None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:13.196024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8a3f2-896e-4afe-8b8d-15412d8b024d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:13.197913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 180/377] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:12 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:13.204271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de1bb1-db98-4331-90d6-fb46f9ef5eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:13.205436 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8a3f2-896e-4afe-8b8d-15412d8b024d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:13.215534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de1bb1-db98-4331-90d6-fb46f9ef5eaa None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:13.216039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de1bb1-db98-4331-90d6-fb46f9ef5eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:13.216552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de1bb1-db98-4331-90d6-fb46f9ef5eaa None admin] PATH_INFO: `/v3/groups/8db249922bd6476eb737d3231ac7d44d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:13.228327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de1bb1-db98-4331-90d6-fb46f9ef5eaa None admin] RBAC: Authorizing `identity:get_group(group_id=8db249922bd6476eb737d3231ac7d44d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:13.230382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de1bb1-db98-4331-90d6-fb46f9ef5eaa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:13.233285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 198/378] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:12 2022] GET /identity/v3/groups/8db249922bd6476eb737d3231ac7d44d => generated 229 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:13.239704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de1bb1-db98-4331-90d6-fb46f9ef5eaa None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:13.395039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8a3f2-896e-4afe-8b8d-15412d8b024d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:13.405028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de1bb1-db98-4331-90d6-fb46f9ef5eaa None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:13.431976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2eb059-5a0f-4601-8d61-e1aff829935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:13.433043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eb059-5a0f-4601-8d61-e1aff829935e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:13.433568 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eb059-5a0f-4601-8d61-e1aff829935e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:13.433835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eb059-5a0f-4601-8d61-e1aff829935e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:13.434818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2eb059-5a0f-4601-8d61-e1aff829935e None admin] RBAC: Adding query filter params (group.id=1e773e646bd7466eb00fe330aab338f2, role.id=e2c9006dc77f473f8e1064f285400a98, scope.project.id=8c32c58c2df149f4842efa86dea7e18f) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:13.435270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2eb059-5a0f-4601-8d61-e1aff829935e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:13.437094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2eb059-5a0f-4601-8d61-e1aff829935e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:13.443659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 181/379] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jul 11 20:44:13 2022] GET /identity/v3/role_assignments?group.id=1e773e646bd7466eb00fe330aab338f2&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=e2c9006dc77f473f8e1064f285400a98 => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:13.443815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d54f8-a09b-4539-ba2c-8131a6b54982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:13.444930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d54f8-a09b-4539-ba2c-8131a6b54982 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:13.445218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d54f8-a09b-4539-ba2c-8131a6b54982 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:13.445495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d54f8-a09b-4539-ba2c-8131a6b54982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:13.445947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb99ee-f299-4f78-8dad-bfa3804fa70f None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:13.446362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d54f8-a09b-4539-ba2c-8131a6b54982 None admin] RBAC: Adding query filter params (group.id=8db249922bd6476eb737d3231ac7d44d, role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=27db080494394cc5b886636221f8edb9) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:13.446467 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb99ee-f299-4f78-8dad-bfa3804fa70f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:13.446670 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d54f8-a09b-4539-ba2c-8131a6b54982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:13.447000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb99ee-f299-4f78-8dad-bfa3804fa70f None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:13.448198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d54f8-a09b-4539-ba2c-8131a6b54982 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:13.453552 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 182/380] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:13.455635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34dd12b-9066-4c96-baa4-bc7a461f95a6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:13.456066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34dd12b-9066-4c96-baa4-bc7a461f95a6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:13.456431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 199/381] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jul 11 20:44:13 2022] GET /identity/v3/role_assignments?group.id=8db249922bd6476eb737d3231ac7d44d&scope.project.id=27db080494394cc5b886636221f8edb9&role.id=60bd1c44ce154338a7face8949d07281 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:13.456743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34dd12b-9066-4c96-baa4-bc7a461f95a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:13.475634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c34dd12b-9066-4c96-baa4-bc7a461f95a6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:13.477139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c34dd12b-9066-4c96-baa4-bc7a461f95a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:13.479450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:13.560270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 183/382] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:13.578999 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d54f8-a09b-4539-ba2c-8131a6b54982 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:13.582097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:13.710085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d54f8-a09b-4539-ba2c-8131a6b54982 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:13.734793 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86f42a-ba0c-44f9-ba90-919320725ae2 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:13.735076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86f42a-ba0c-44f9-ba90-919320725ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:13.735348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86f42a-ba0c-44f9-ba90-919320725ae2 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:13.738206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 184/383] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:13.742518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4aec6-ef9b-4f08-8707-d538d93f43c6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:13.742804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4aec6-ef9b-4f08-8707-d538d93f43c6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:13.743076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4aec6-ef9b-4f08-8707-d538d93f43c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:13.746886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee76db8-7cc4-44a3-bda5-43fb7cedd42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:13.747887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee76db8-7cc4-44a3-bda5-43fb7cedd42b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:13.748348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee76db8-7cc4-44a3-bda5-43fb7cedd42b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:13.748770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee76db8-7cc4-44a3-bda5-43fb7cedd42b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:13.762505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-fec4aec6-ef9b-4f08-8707-d538d93f43c6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:13.762784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee76db8-7cc4-44a3-bda5-43fb7cedd42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:13.763680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fec4aec6-ef9b-4f08-8707-d538d93f43c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-4ee76db8-7cc4-44a3-bda5-43fb7cedd42b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:13.771591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:13.772845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:13.773808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:13.776207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 200/384] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:13.782431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee76db8-7cc4-44a3-bda5-43fb7cedd42b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:13.885650 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 185/385] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:13.920520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:13.924959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4aec6-ef9b-4f08-8707-d538d93f43c6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:13.948265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee76db8-7cc4-44a3-bda5-43fb7cedd42b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:14.018959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4ac58-790a-41c3-a147-0a0f804d9b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:14.019582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4ac58-790a-41c3-a147-0a0f804d9b79 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.019863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4ac58-790a-41c3-a147-0a0f804d9b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.021361 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4ac58-790a-41c3-a147-0a0f804d9b79 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.022148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4ac58-790a-41c3-a147-0a0f804d9b79 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:14.022487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4ac58-790a-41c3-a147-0a0f804d9b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:14.023912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4ac58-790a-41c3-a147-0a0f804d9b79 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:14.043641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 201/386] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:14.045283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:14.050592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4ac58-790a-41c3-a147-0a0f804d9b79 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:14.108066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4aec6-ef9b-4f08-8707-d538d93f43c6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:14.162937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f576d3-f21e-4072-8ea2-4cd12f1259e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:14.164099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f576d3-f21e-4072-8ea2-4cd12f1259e6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.164550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f576d3-f21e-4072-8ea2-4cd12f1259e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.165051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f576d3-f21e-4072-8ea2-4cd12f1259e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.177728 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4ac58-790a-41c3-a147-0a0f804d9b79 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:14.185797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f576d3-f21e-4072-8ea2-4cd12f1259e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-05f576d3-f21e-4072-8ea2-4cd12f1259e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:14.197985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:14.198984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:14.199741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:14.205579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 186/387] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:44:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:14.205793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:14.213474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f576d3-f21e-4072-8ea2-4cd12f1259e6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:14.215057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29439147-53ee-4d1f-bbd5-ce8bfa22aff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:14.215951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29439147-53ee-4d1f-bbd5-ce8bfa22aff6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.216381 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29439147-53ee-4d1f-bbd5-ce8bfa22aff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.216667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29439147-53ee-4d1f-bbd5-ce8bfa22aff6 None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.218086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29439147-53ee-4d1f-bbd5-ce8bfa22aff6 None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:14.220067 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29439147-53ee-4d1f-bbd5-ce8bfa22aff6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:14.222059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 202/388] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:14 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:14.227612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29439147-53ee-4d1f-bbd5-ce8bfa22aff6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:14.339455 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29439147-53ee-4d1f-bbd5-ce8bfa22aff6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:14.374206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f576d3-f21e-4072-8ea2-4cd12f1259e6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:14.379388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9a1d5-7031-4cdc-b061-b916fd0d221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:14.380349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9a1d5-7031-4cdc-b061-b916fd0d221b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.380637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9a1d5-7031-4cdc-b061-b916fd0d221b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.380918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9a1d5-7031-4cdc-b061-b916fd0d221b None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.382512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9a1d5-7031-4cdc-b061-b916fd0d221b None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:14.384028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9a1d5-7031-4cdc-b061-b916fd0d221b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:14.386321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 203/389] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:14 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:14.387650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b651a2-9922-43b6-9b92-dde5dfdc8391 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.387940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b651a2-9922-43b6-9b92-dde5dfdc8391 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.391731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b651a2-9922-43b6-9b92-dde5dfdc8391 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.393335 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 204/390] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:14.395485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b651a2-9922-43b6-9b92-dde5dfdc8391 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:14.401642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:14.411530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1b78f-3d01-4403-b7f1-d112b65d49f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:14.412340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b78f-3d01-4403-b7f1-d112b65d49f3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.412703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b78f-3d01-4403-b7f1-d112b65d49f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.413243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b78f-3d01-4403-b7f1-d112b65d49f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.414088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1b78f-3d01-4403-b7f1-d112b65d49f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:14.414808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1b78f-3d01-4403-b7f1-d112b65d49f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:14.416697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1b78f-3d01-4403-b7f1-d112b65d49f3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:14.430452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 187/391] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:14.431904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d0e77-1198-4141-9383-a14d7776322a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.432211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d0e77-1198-4141-9383-a14d7776322a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.432488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d0e77-1198-4141-9383-a14d7776322a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.450592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5d3d0e77-1198-4141-9383-a14d7776322a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:14.451772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5d3d0e77-1198-4141-9383-a14d7776322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:14.467056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:14.514132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b651a2-9922-43b6-9b92-dde5dfdc8391 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:14.523894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 188/392] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:14.524968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d0e77-1198-4141-9383-a14d7776322a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:14.550758 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb53845-faf1-4503-8645-60c4e7d48318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:14.551712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53845-faf1-4503-8645-60c4e7d48318 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.552215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53845-faf1-4503-8645-60c4e7d48318 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.552548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53845-faf1-4503-8645-60c4e7d48318 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3/groups/1e773e646bd7466eb00fe330aab338f2/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.555227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb53845-faf1-4503-8645-60c4e7d48318 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad2772943334037bb33953f59d816e3, group_id=1e773e646bd7466eb00fe330aab338f2, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:14.556872 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb53845-faf1-4503-8645-60c4e7d48318 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:14.576940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 205/393] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Jul 11 20:44:14 2022] PUT /identity/v3/projects/3ad2772943334037bb33953f59d816e3/groups/1e773e646bd7466eb00fe330aab338f2/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:14.577838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb53845-faf1-4503-8645-60c4e7d48318 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:14.630233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d0e77-1198-4141-9383-a14d7776322a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:14.652162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:14.668775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490260b9-b888-47e5-83c3-648ce72f61cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:14.669797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490260b9-b888-47e5-83c3-648ce72f61cd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.670171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490260b9-b888-47e5-83c3-648ce72f61cd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.670537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490260b9-b888-47e5-83c3-648ce72f61cd None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.671842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490260b9-b888-47e5-83c3-648ce72f61cd None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:14.673434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490260b9-b888-47e5-83c3-648ce72f61cd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:14.680047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 189/394] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:14 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:14.681361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490260b9-b888-47e5-83c3-648ce72f61cd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:14.718092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb53845-faf1-4503-8645-60c4e7d48318 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:14.755698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:14.760353 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc1a23-cd66-4e79-9c39-bead6e2a7b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:14.761229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc1a23-cd66-4e79-9c39-bead6e2a7b59 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.761517 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc1a23-cd66-4e79-9c39-bead6e2a7b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.761915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc1a23-cd66-4e79-9c39-bead6e2a7b59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.777201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc1a23-cd66-4e79-9c39-bead6e2a7b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9fbc1a23-cd66-4e79-9c39-bead6e2a7b59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:14.785743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:14.786629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:14.787433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 206/395] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:14.793394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc1a23-cd66-4e79-9c39-bead6e2a7b59 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:14.794476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490260b9-b888-47e5-83c3-648ce72f61cd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:14.839370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423564e-5c14-45f9-a438-db08930f3c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:14.840595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423564e-5c14-45f9-a438-db08930f3c02 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.840876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423564e-5c14-45f9-a438-db08930f3c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.842160 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423564e-5c14-45f9-a438-db08930f3c02 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.844954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423564e-5c14-45f9-a438-db08930f3c02 None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:14.846694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423564e-5c14-45f9-a438-db08930f3c02 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:14.854402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 190/396] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:14 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:14.856313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423564e-5c14-45f9-a438-db08930f3c02 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:14.935299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc1a23-cd66-4e79-9c39-bead6e2a7b59 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:14.943035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:14.983287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca9665-add1-4814-8e8e-690810fd87d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:14.985249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca9665-add1-4814-8e8e-690810fd87d4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:14.985538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca9665-add1-4814-8e8e-690810fd87d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:14.985816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca9665-add1-4814-8e8e-690810fd87d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:14.986583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca9665-add1-4814-8e8e-690810fd87d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:14.986895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca9665-add1-4814-8e8e-690810fd87d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:14.988440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca9665-add1-4814-8e8e-690810fd87d4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:14.998453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 207/397] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:14.999791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423564e-5c14-45f9-a438-db08930f3c02 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:15.003549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca9665-add1-4814-8e8e-690810fd87d4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:15.042556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed2bc8-79df-44a2-94b0-20c09efd174d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:15.044015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed2bc8-79df-44a2-94b0-20c09efd174d None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:15.044843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed2bc8-79df-44a2-94b0-20c09efd174d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:15.045536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed2bc8-79df-44a2-94b0-20c09efd174d None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3/groups/1e773e646bd7466eb00fe330aab338f2/roles/e2c9006dc77f473f8e1064f285400a98` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:15.049165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed2bc8-79df-44a2-94b0-20c09efd174d None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad2772943334037bb33953f59d816e3, group_id=1e773e646bd7466eb00fe330aab338f2, role_id=e2c9006dc77f473f8e1064f285400a98)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:15.050826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed2bc8-79df-44a2-94b0-20c09efd174d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:15.058903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:15.076813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 191/398] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Jul 11 20:44:14 2022] PUT /identity/v3/projects/3ad2772943334037bb33953f59d816e3/groups/1e773e646bd7466eb00fe330aab338f2/roles/e2c9006dc77f473f8e1064f285400a98 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:15.132637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca9665-add1-4814-8e8e-690810fd87d4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:15.170340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd7c72-c65e-4894-a6d8-dbe154c30e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:15.171283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd7c72-c65e-4894-a6d8-dbe154c30e33 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:15.171567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd7c72-c65e-4894-a6d8-dbe154c30e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:15.171842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd7c72-c65e-4894-a6d8-dbe154c30e33 None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:15.173159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd7c72-c65e-4894-a6d8-dbe154c30e33 None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:15.174728 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd7c72-c65e-4894-a6d8-dbe154c30e33 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:15.176895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 208/399] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:15 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:15.182807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed2bc8-79df-44a2-94b0-20c09efd174d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:15.293180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed2bc8-79df-44a2-94b0-20c09efd174d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:15.320521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:15.331187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42ed75-c032-4a6a-a876-ec1162d88ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:15.332535 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42ed75-c032-4a6a-a876-ec1162d88ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:15.332823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42ed75-c032-4a6a-a876-ec1162d88ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:15.333107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42ed75-c032-4a6a-a876-ec1162d88ef3 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:15.334687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42ed75-c032-4a6a-a876-ec1162d88ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:15.336283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42ed75-c032-4a6a-a876-ec1162d88ef3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:15.338351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 192/400] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:15 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:15.344688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd7c72-c65e-4894-a6d8-dbe154c30e33 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:15.454499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd7c72-c65e-4894-a6d8-dbe154c30e33 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:15.465448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:15.491069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f98a7-5979-46d5-924a-4b26a42d8809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:15.492433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f98a7-5979-46d5-924a-4b26a42d8809 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:15.492717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f98a7-5979-46d5-924a-4b26a42d8809 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:15.492998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f98a7-5979-46d5-924a-4b26a42d8809 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f/groups/1e773e646bd7466eb00fe330aab338f2/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:15.495115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f98a7-5979-46d5-924a-4b26a42d8809 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c32c58c2df149f4842efa86dea7e18f, group_id=1e773e646bd7466eb00fe330aab338f2, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:15.496604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f98a7-5979-46d5-924a-4b26a42d8809 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:15.515391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 209/401] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Jul 11 20:44:15 2022] PUT /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f/groups/1e773e646bd7466eb00fe330aab338f2/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:16.122580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8702447-8be3-4974-8149-c4024de1f17f None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:16.123120 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8702447-8be3-4974-8149-c4024de1f17f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:16.123120 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8702447-8be3-4974-8149-c4024de1f17f None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:16.124832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 193/402] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:16.130422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e92469-3a52-4737-82f3-f7159dd426dd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:16.130842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e92469-3a52-4737-82f3-f7159dd426dd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:16.131027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e92469-3a52-4737-82f3-f7159dd426dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:16.150462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-05e92469-3a52-4737-82f3-f7159dd426dd None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:16.151663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-05e92469-3a52-4737-82f3-f7159dd426dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:16.231547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 210/403] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:16.256607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3b7ef-f744-4567-b25a-b51dbafde594 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:16.257554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3b7ef-f744-4567-b25a-b51dbafde594 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:16.257554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3b7ef-f744-4567-b25a-b51dbafde594 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:16.258994 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 194/404] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:16.264543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566a00-24ad-49d7-84dd-32f25921d9b1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:16.264892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566a00-24ad-49d7-84dd-32f25921d9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:16.265175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566a00-24ad-49d7-84dd-32f25921d9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:16.265292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3b7ef-f744-4567-b25a-b51dbafde594 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:16.282479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d3566a00-24ad-49d7-84dd-32f25921d9b1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:16.284068 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d3566a00-24ad-49d7-84dd-32f25921d9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:16.399219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 211/405] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:16.430986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566a00-24ad-49d7-84dd-32f25921d9b1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:16.458848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3b7ef-f744-4567-b25a-b51dbafde594 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:16.479448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:16.529687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e54c5-1937-43d5-8b1a-b4b5811a02ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:16.530519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e54c5-1937-43d5-8b1a-b4b5811a02ec None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:16.530800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e54c5-1937-43d5-8b1a-b4b5811a02ec None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:16.531072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e54c5-1937-43d5-8b1a-b4b5811a02ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:16.557173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e54c5-1937-43d5-8b1a-b4b5811a02ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-288e54c5-1937-43d5-8b1a-b4b5811a02ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:16.565689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:16.566549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:16.567343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:16.572869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 195/406] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:44:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:16.578665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e54c5-1937-43d5-8b1a-b4b5811a02ec None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:16.585032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566a00-24ad-49d7-84dd-32f25921d9b1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:16.639166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:16.641425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e5eb8-c47e-42f0-8522-bce55be3f68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:16.642257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e5eb8-c47e-42f0-8522-bce55be3f68f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:16.642542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e5eb8-c47e-42f0-8522-bce55be3f68f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:16.642815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e5eb8-c47e-42f0-8522-bce55be3f68f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:16.673114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e5eb8-c47e-42f0-8522-bce55be3f68f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b76e5eb8-c47e-42f0-8522-bce55be3f68f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:16.681689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:16.682576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:16.683341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:16.691005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 212/407] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:16.695824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e5eb8-c47e-42f0-8522-bce55be3f68f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:16.700492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e54c5-1937-43d5-8b1a-b4b5811a02ec None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:16.738608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff0e12-bf06-4aea-8e04-94a18a2691fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:16.739304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff0e12-bf06-4aea-8e04-94a18a2691fc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:16.739636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff0e12-bf06-4aea-8e04-94a18a2691fc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:16.739918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff0e12-bf06-4aea-8e04-94a18a2691fc None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:16.744133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff0e12-bf06-4aea-8e04-94a18a2691fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:16.744133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff0e12-bf06-4aea-8e04-94a18a2691fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:16.744133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff0e12-bf06-4aea-8e04-94a18a2691fc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:16.754130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 196/408] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:16.759488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff0e12-bf06-4aea-8e04-94a18a2691fc None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:16.844582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:16.863767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e5eb8-c47e-42f0-8522-bce55be3f68f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:16.930159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95303920-a6c5-4bcf-bb0a-bc83a259d4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:16.930772 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303920-a6c5-4bcf-bb0a-bc83a259d4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:16.931057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303920-a6c5-4bcf-bb0a-bc83a259d4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:16.931437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303920-a6c5-4bcf-bb0a-bc83a259d4f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:16.932243 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95303920-a6c5-4bcf-bb0a-bc83a259d4f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:16.932556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95303920-a6c5-4bcf-bb0a-bc83a259d4f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:16.934023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95303920-a6c5-4bcf-bb0a-bc83a259d4f0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:16.944880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff0e12-bf06-4aea-8e04-94a18a2691fc None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:16.952782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:16.953756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 213/409] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:16.959240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95303920-a6c5-4bcf-bb0a-bc83a259d4f0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:16.980637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3eba1-f536-4579-a090-d637718d99ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:16.981513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3eba1-f536-4579-a090-d637718d99ff None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:16.981791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3eba1-f536-4579-a090-d637718d99ff None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:16.982065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3eba1-f536-4579-a090-d637718d99ff None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:16.983251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3eba1-f536-4579-a090-d637718d99ff None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:16.984747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3eba1-f536-4579-a090-d637718d99ff None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:16.986729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 197/410] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:16 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:16.996904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3eba1-f536-4579-a090-d637718d99ff None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:17.144830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95303920-a6c5-4bcf-bb0a-bc83a259d4f0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:17.194712 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3eba1-f536-4579-a090-d637718d99ff None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:17.206802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd1d87-3136-4246-a482-8e87bc7f3c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:17.208002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd1d87-3136-4246-a482-8e87bc7f3c9d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:17.208498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd1d87-3136-4246-a482-8e87bc7f3c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:17.208961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd1d87-3136-4246-a482-8e87bc7f3c9d None admin] PATH_INFO: `/v3/groups/8db249922bd6476eb737d3231ac7d44d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:17.211207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd1d87-3136-4246-a482-8e87bc7f3c9d None admin] RBAC: Authorizing `identity:get_group(group_id=8db249922bd6476eb737d3231ac7d44d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:17.214535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd1d87-3136-4246-a482-8e87bc7f3c9d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:17.219071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 214/411] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:16 2022] GET /identity/v3/groups/8db249922bd6476eb737d3231ac7d44d => generated 229 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:17.222029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd1d87-3136-4246-a482-8e87bc7f3c9d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:17.266667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c123e94-75ee-4b74-8e38-d2e4d6b0c377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:17.268099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c123e94-75ee-4b74-8e38-d2e4d6b0c377 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:17.268494 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:17.268652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c123e94-75ee-4b74-8e38-d2e4d6b0c377 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:17.269102 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c123e94-75ee-4b74-8e38-d2e4d6b0c377 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:17.273422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c123e94-75ee-4b74-8e38-d2e4d6b0c377 None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:17.275742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c123e94-75ee-4b74-8e38-d2e4d6b0c377 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:17.280234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 198/412] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:16 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:17.284986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c123e94-75ee-4b74-8e38-d2e4d6b0c377 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:17.370330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:17.401984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd1d87-3136-4246-a482-8e87bc7f3c9d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:17.449599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18928993-c302-40f9-a787-e08d3e5ebf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:17.451065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18928993-c302-40f9-a787-e08d3e5ebf79 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:17.451504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18928993-c302-40f9-a787-e08d3e5ebf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:17.451967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18928993-c302-40f9-a787-e08d3e5ebf79 None admin] PATH_INFO: `/v3/projects/27db080494394cc5b886636221f8edb9` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:17.453413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18928993-c302-40f9-a787-e08d3e5ebf79 None admin] RBAC: Authorizing `identity:get_project(project_id=27db080494394cc5b886636221f8edb9)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:17.455010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18928993-c302-40f9-a787-e08d3e5ebf79 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:17.455465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c123e94-75ee-4b74-8e38-d2e4d6b0c377 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:17.457531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 215/413] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:17 2022] GET /identity/v3/projects/27db080494394cc5b886636221f8edb9 => generated 344 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:17.459058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75ae47-d49f-4853-bed0-3d12afd49d46 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:17.459697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75ae47-d49f-4853-bed0-3d12afd49d46 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:17.460415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75ae47-d49f-4853-bed0-3d12afd49d46 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:17.462366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 216/414] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:17.463272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75ae47-d49f-4853-bed0-3d12afd49d46 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:17.497134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4f171-88ee-438a-88ed-f3ce877fc726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:17.498340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4f171-88ee-438a-88ed-f3ce877fc726 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:17.498762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4f171-88ee-438a-88ed-f3ce877fc726 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:17.499206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4f171-88ee-438a-88ed-f3ce877fc726 None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f/groups/1e773e646bd7466eb00fe330aab338f2/roles/e2c9006dc77f473f8e1064f285400a98` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:17.501726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4f171-88ee-438a-88ed-f3ce877fc726 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c32c58c2df149f4842efa86dea7e18f, group_id=1e773e646bd7466eb00fe330aab338f2, role_id=e2c9006dc77f473f8e1064f285400a98)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:17.503348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4f171-88ee-438a-88ed-f3ce877fc726 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:17.520558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 199/415] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Jul 11 20:44:17 2022] PUT /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f/groups/1e773e646bd7466eb00fe330aab338f2/roles/e2c9006dc77f473f8e1064f285400a98 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:17.523181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f20f5f-b33b-4d27-aa04-5bf0668bae2f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:17.523706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f20f5f-b33b-4d27-aa04-5bf0668bae2f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:17.524188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f20f5f-b33b-4d27-aa04-5bf0668bae2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:17.544790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a2f20f5f-b33b-4d27-aa04-5bf0668bae2f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:17.546380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a2f20f5f-b33b-4d27-aa04-5bf0668bae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:17.630605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 200/416] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:17.661706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:17.664275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f20f5f-b33b-4d27-aa04-5bf0668bae2f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:17.697887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75ae47-d49f-4853-bed0-3d12afd49d46 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:17.707560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:17.752847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae55984-7d07-43b0-955e-62a7410ea55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:17.754516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae55984-7d07-43b0-955e-62a7410ea55b None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:17.754802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae55984-7d07-43b0-955e-62a7410ea55b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:17.755086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae55984-7d07-43b0-955e-62a7410ea55b None admin] PATH_INFO: `/v3/projects/27db080494394cc5b886636221f8edb9/groups/8db249922bd6476eb737d3231ac7d44d/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:17.757880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae55984-7d07-43b0-955e-62a7410ea55b None admin] RBAC: Authorizing `identity:create_grant(project_id=27db080494394cc5b886636221f8edb9, group_id=8db249922bd6476eb737d3231ac7d44d, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:17.759001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae55984-7d07-43b0-955e-62a7410ea55b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:17.778159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 217/417] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Jul 11 20:44:17 2022] PUT /identity/v3/projects/27db080494394cc5b886636221f8edb9/groups/8db249922bd6476eb737d3231ac7d44d/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:17.801439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f20f5f-b33b-4d27-aa04-5bf0668bae2f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:17.844704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2531e-b026-4397-9738-0337b994c4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:17.845536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2531e-b026-4397-9738-0337b994c4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:17.845825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2531e-b026-4397-9738-0337b994c4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:17.846101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2531e-b026-4397-9738-0337b994c4c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:17.860348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2531e-b026-4397-9738-0337b994c4c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a3a2531e-b026-4397-9738-0337b994c4c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:17.869295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:17.870319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:17.871134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 201/418] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:17.876837 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae55984-7d07-43b0-955e-62a7410ea55b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:18.001099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae55984-7d07-43b0-955e-62a7410ea55b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:18.034856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb71c0-6b2b-482e-90e9-1a5ff875d03f None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.035187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb71c0-6b2b-482e-90e9-1a5ff875d03f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.035420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb71c0-6b2b-482e-90e9-1a5ff875d03f None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.038868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 202/419] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:44:18.045824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e786b-2761-42fd-98e9-411979ae79b1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.046123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e786b-2761-42fd-98e9-411979ae79b1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.046486 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e786b-2761-42fd-98e9-411979ae79b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.052037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da52d4c-39ab-43c1-bec4-5378a38c4e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:18.054255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da52d4c-39ab-43c1-bec4-5378a38c4e34 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.056363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da52d4c-39ab-43c1-bec4-5378a38c4e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.057668 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da52d4c-39ab-43c1-bec4-5378a38c4e34 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.058642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da52d4c-39ab-43c1-bec4-5378a38c4e34 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:18.059142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da52d4c-39ab-43c1-bec4-5378a38c4e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:18.063874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da52d4c-39ab-43c1-bec4-5378a38c4e34 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:18.071989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-738e786b-2761-42fd-98e9-411979ae79b1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:18.073223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-738e786b-2761-42fd-98e9-411979ae79b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:18.074032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 218/420] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:18.080011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da52d4c-39ab-43c1-bec4-5378a38c4e34 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:18.093708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:18.158432 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 203/421] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:18.184662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e786b-2761-42fd-98e9-411979ae79b1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:18.197601 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da52d4c-39ab-43c1-bec4-5378a38c4e34 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:18.217688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:18.263402 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb47b0f-e987-42bb-be53-ce6126409e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:18.264343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb47b0f-e987-42bb-be53-ce6126409e2f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.264765 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb47b0f-e987-42bb-be53-ce6126409e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.265176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb47b0f-e987-42bb-be53-ce6126409e2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.330078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e786b-2761-42fd-98e9-411979ae79b1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:18.346245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb47b0f-e987-42bb-be53-ce6126409e2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:18.347954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb47b0f-e987-42bb-be53-ce6126409e2f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:18.360720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:18.375602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd58367-a4f8-4193-8105-8b648acf4ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:18.376489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58367-a4f8-4193-8105-8b648acf4ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.377162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58367-a4f8-4193-8105-8b648acf4ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.377443 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58367-a4f8-4193-8105-8b648acf4ef8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.391496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd58367-a4f8-4193-8105-8b648acf4ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-1cd58367-a4f8-4193-8105-8b648acf4ef8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:18.402484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:18.403494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:18.404362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:18.408204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 204/422] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:44:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:18.410916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a8555-39b8-484c-811f-9af82743b807 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.411198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a8555-39b8-484c-811f-9af82743b807 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.411469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a8555-39b8-484c-811f-9af82743b807 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.412907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 205/423] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:18.418172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a8555-39b8-484c-811f-9af82743b807 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:18.480213 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 219/424] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 20:44:18.482023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae75f71-a3ce-4753-a67c-0801880a9b92 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.482736 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae75f71-a3ce-4753-a67c-0801880a9b92 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.484487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae75f71-a3ce-4753-a67c-0801880a9b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.506496 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2ae75f71-a3ce-4753-a67c-0801880a9b92 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:18.507970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2ae75f71-a3ce-4753-a67c-0801880a9b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:18.519175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:18.540835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a8555-39b8-484c-811f-9af82743b807 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:18.577526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574539a2-166e-46a6-914c-1facc4b16584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:18.578685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574539a2-166e-46a6-914c-1facc4b16584 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.579174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574539a2-166e-46a6-914c-1facc4b16584 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.579630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574539a2-166e-46a6-914c-1facc4b16584 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.580761 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574539a2-166e-46a6-914c-1facc4b16584 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:18.581301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574539a2-166e-46a6-914c-1facc4b16584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:18.582959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574539a2-166e-46a6-914c-1facc4b16584 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:18.592429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 206/425] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:18.593526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574539a2-166e-46a6-914c-1facc4b16584 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:18.604143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 220/426] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:18.604143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae75f71-a3ce-4753-a67c-0801880a9b92 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:18.644537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:18.715794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae75f71-a3ce-4753-a67c-0801880a9b92 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:18.727841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574539a2-166e-46a6-914c-1facc4b16584 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:18.758259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756b14e-2a66-4ce7-9d8e-5ec773647042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:18.758952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756b14e-2a66-4ce7-9d8e-5ec773647042 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.759235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756b14e-2a66-4ce7-9d8e-5ec773647042 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.759644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756b14e-2a66-4ce7-9d8e-5ec773647042 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.766737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0997139-7857-48bf-9674-06348e8d54aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:18.767879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0997139-7857-48bf-9674-06348e8d54aa None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.768389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0997139-7857-48bf-9674-06348e8d54aa None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.768955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0997139-7857-48bf-9674-06348e8d54aa None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.770411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0997139-7857-48bf-9674-06348e8d54aa None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:18.772244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0997139-7857-48bf-9674-06348e8d54aa None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:18.775047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 207/427] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:18 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:18.776128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0997139-7857-48bf-9674-06348e8d54aa None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:18.793052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:18.841574 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756b14e-2a66-4ce7-9d8e-5ec773647042 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:18.843298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756b14e-2a66-4ce7-9d8e-5ec773647042 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:18.889468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:18.906050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0997139-7857-48bf-9674-06348e8d54aa None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:18.947886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557fdf7-c516-40a6-9203-bdc3771fcef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:18.948869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557fdf7-c516-40a6-9203-bdc3771fcef4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:18.949242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557fdf7-c516-40a6-9203-bdc3771fcef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:18.949516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557fdf7-c516-40a6-9203-bdc3771fcef4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:18.957422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 221/428] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 20:44:18.958558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756b14e-2a66-4ce7-9d8e-5ec773647042 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:18.967750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557fdf7-c516-40a6-9203-bdc3771fcef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9557fdf7-c516-40a6-9203-bdc3771fcef4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:18.977914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:18.978776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 20:44:18.979576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:18.984344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 208/429] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:18.985176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557fdf7-c516-40a6-9203-bdc3771fcef4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:19.062322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:19.095722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756b14e-2a66-4ce7-9d8e-5ec773647042 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:19.096704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557fdf7-c516-40a6-9203-bdc3771fcef4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:19.134124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8bd9b-8988-4ba9-8cc2-27277aca85a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:19.135620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8bd9b-8988-4ba9-8cc2-27277aca85a6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:19.136097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8bd9b-8988-4ba9-8cc2-27277aca85a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:19.136625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8bd9b-8988-4ba9-8cc2-27277aca85a6 None admin] PATH_INFO: `/v3/projects/3ad2772943334037bb33953f59d816e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:19.138160 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8bd9b-8988-4ba9-8cc2-27277aca85a6 None admin] RBAC: Authorizing `identity:get_project(project_id=3ad2772943334037bb33953f59d816e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:19.139608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a8ec1-9de9-4d7b-aa23-f07c805802ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:19.140144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8bd9b-8988-4ba9-8cc2-27277aca85a6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:19.140535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a8ec1-9de9-4d7b-aa23-f07c805802ae None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:19.140817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a8ec1-9de9-4d7b-aa23-f07c805802ae None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:19.141124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a8ec1-9de9-4d7b-aa23-f07c805802ae None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:19.142362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a8ec1-9de9-4d7b-aa23-f07c805802ae None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:19.143469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 209/430] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:18 2022] GET /identity/v3/projects/3ad2772943334037bb33953f59d816e3 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:19.143860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a8ec1-9de9-4d7b-aa23-f07c805802ae None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:19.144746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8bd9b-8988-4ba9-8cc2-27277aca85a6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:19.145740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 222/431] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:18 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:19.152428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a8ec1-9de9-4d7b-aa23-f07c805802ae None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:19.165222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:19.265210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a8ec1-9de9-4d7b-aa23-f07c805802ae None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:19.272377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8bd9b-8988-4ba9-8cc2-27277aca85a6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:19.301795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1193b34-6586-4413-a7e0-f952f4d9124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:19.302914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1193b34-6586-4413-a7e0-f952f4d9124f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:19.303549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1193b34-6586-4413-a7e0-f952f4d9124f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:19.304021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1193b34-6586-4413-a7e0-f952f4d9124f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:19.305061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1193b34-6586-4413-a7e0-f952f4d9124f None admin] RBAC: Adding query filter params (group.id=1e773e646bd7466eb00fe330aab338f2, role.id=1403cb7e4e264c68b63e467a2e433699, scope.project.id=3ad2772943334037bb33953f59d816e3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:19.305575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1193b34-6586-4413-a7e0-f952f4d9124f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:19.306974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fafe1-305c-49da-8070-769994170c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:19.307412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1193b34-6586-4413-a7e0-f952f4d9124f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:19.307649 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fafe1-305c-49da-8070-769994170c47 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:19.307932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fafe1-305c-49da-8070-769994170c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:19.308238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fafe1-305c-49da-8070-769994170c47 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:19.308997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fafe1-305c-49da-8070-769994170c47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:19.309309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fafe1-305c-49da-8070-769994170c47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:19.310670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fafe1-305c-49da-8070-769994170c47 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:19.315694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 223/432] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jul 11 20:44:19 2022] GET /identity/v3/role_assignments?group.id=1e773e646bd7466eb00fe330aab338f2&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:19.316757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1193b34-6586-4413-a7e0-f952f4d9124f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:19.328968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 210/433] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:44:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:19.334367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fafe1-305c-49da-8070-769994170c47 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:19.337362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:19.398257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:19.433035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1193b34-6586-4413-a7e0-f952f4d9124f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:19.472357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135e0d-6d96-4026-9418-31ed9d37151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:19.473720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135e0d-6d96-4026-9418-31ed9d37151a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:19.474218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135e0d-6d96-4026-9418-31ed9d37151a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:19.474726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135e0d-6d96-4026-9418-31ed9d37151a None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:19.476913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd135e0d-6d96-4026-9418-31ed9d37151a None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:19.478696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd135e0d-6d96-4026-9418-31ed9d37151a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:19.480640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 224/434] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:19 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:19.489490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135e0d-6d96-4026-9418-31ed9d37151a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:19.528621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fafe1-305c-49da-8070-769994170c47 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:19.606659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b234117-0d74-4113-b002-1c839799b98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:19.607796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234117-0d74-4113-b002-1c839799b98a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:19.608423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234117-0d74-4113-b002-1c839799b98a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:19.608879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234117-0d74-4113-b002-1c839799b98a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:19.662807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135e0d-6d96-4026-9418-31ed9d37151a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:19.693323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b234117-0d74-4113-b002-1c839799b98a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:19.695505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b234117-0d74-4113-b002-1c839799b98a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:19.701838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:19.711731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb155e7-fe39-461d-9f11-bcab4ad364da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:19.712684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb155e7-fe39-461d-9f11-bcab4ad364da None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:19.713106 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb155e7-fe39-461d-9f11-bcab4ad364da None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:19.713898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb155e7-fe39-461d-9f11-bcab4ad364da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:19.715190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb155e7-fe39-461d-9f11-bcab4ad364da None admin] RBAC: Adding query filter params (group.id=1e773e646bd7466eb00fe330aab338f2, role.id=e2c9006dc77f473f8e1064f285400a98, scope.project.id=3ad2772943334037bb33953f59d816e3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:19.715647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb155e7-fe39-461d-9f11-bcab4ad364da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:19.717546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb155e7-fe39-461d-9f11-bcab4ad364da None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:19.729975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 225/435] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jul 11 20:44:19 2022] GET /identity/v3/role_assignments?group.id=1e773e646bd7466eb00fe330aab338f2&scope.project.id=3ad2772943334037bb33953f59d816e3&role.id=e2c9006dc77f473f8e1064f285400a98 => generated 611 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:19.838352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccbdcb-8f5f-42d4-b86a-2bec94c31502 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:19.838765 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccbdcb-8f5f-42d4-b86a-2bec94c31502 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:19.839042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccbdcb-8f5f-42d4-b86a-2bec94c31502 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:19.840739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 226/436] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:44:19.846149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa90f1b-754c-45bf-a085-9190c875b7a4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:19.846463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa90f1b-754c-45bf-a085-9190c875b7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:19.846703 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa90f1b-754c-45bf-a085-9190c875b7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:19.851454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 211/437] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 20:44:19.857729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:19.858046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b234117-0d74-4113-b002-1c839799b98a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:19.875393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1fa90f1b-754c-45bf-a085-9190c875b7a4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:19.877008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1fa90f1b-754c-45bf-a085-9190c875b7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:19.968489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 227/438] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:19.985362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b234117-0d74-4113-b002-1c839799b98a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:19.987423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:19.996082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa90f1b-754c-45bf-a085-9190c875b7a4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:20.027946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0c3ed-f857-4da2-9d85-f911a2d1550a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:20.028953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0c3ed-f857-4da2-9d85-f911a2d1550a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.029328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0c3ed-f857-4da2-9d85-f911a2d1550a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.029609 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0c3ed-f857-4da2-9d85-f911a2d1550a None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.030950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0c3ed-f857-4da2-9d85-f911a2d1550a None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:20.032581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0c3ed-f857-4da2-9d85-f911a2d1550a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:20.039438 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 212/439] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:19 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:20.040368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0c3ed-f857-4da2-9d85-f911a2d1550a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:20.121746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa90f1b-754c-45bf-a085-9190c875b7a4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:20.162083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6f57a-b4af-4f51-97c2-62350ba2af33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:20.162999 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6f57a-b4af-4f51-97c2-62350ba2af33 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.163281 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6f57a-b4af-4f51-97c2-62350ba2af33 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.163538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0c3ed-f857-4da2-9d85-f911a2d1550a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:20.163632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6f57a-b4af-4f51-97c2-62350ba2af33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.168520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:20.178063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6f57a-b4af-4f51-97c2-62350ba2af33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-39c6f57a-b4af-4f51-97c2-62350ba2af33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:20.186392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:20.187276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:20.188054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 228/440] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:44:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:20.190160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e484d2-7e0f-4206-87ed-7f1988fef485 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.190435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e484d2-7e0f-4206-87ed-7f1988fef485 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.190717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e484d2-7e0f-4206-87ed-7f1988fef485 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.192004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 229/441] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:20.194146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e484d2-7e0f-4206-87ed-7f1988fef485 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:20.207051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6842b9-014f-467b-8134-bfad9b7ce94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:20.207955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6842b9-014f-467b-8134-bfad9b7ce94e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.208365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6842b9-014f-467b-8134-bfad9b7ce94e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.208642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6842b9-014f-467b-8134-bfad9b7ce94e None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.210457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6842b9-014f-467b-8134-bfad9b7ce94e None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:20.211917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6842b9-014f-467b-8134-bfad9b7ce94e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:20.213764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 213/442] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:20 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:20.214767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525d12f-ab23-4d88-9881-f553883c7fd0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.215050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525d12f-ab23-4d88-9881-f553883c7fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.215322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525d12f-ab23-4d88-9881-f553883c7fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.233010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f525d12f-ab23-4d88-9881-f553883c7fd0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:20.234198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f525d12f-ab23-4d88-9881-f553883c7fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:20.274269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:20.319938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 214/443] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:20.320948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525d12f-ab23-4d88-9881-f553883c7fd0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:20.334249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e484d2-7e0f-4206-87ed-7f1988fef485 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:20.369283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa97a4-2044-4aac-9d51-6061cf7112c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:20.369966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa97a4-2044-4aac-9d51-6061cf7112c8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.370251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa97a4-2044-4aac-9d51-6061cf7112c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.370529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa97a4-2044-4aac-9d51-6061cf7112c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.371285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa97a4-2044-4aac-9d51-6061cf7112c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:20.371595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa97a4-2044-4aac-9d51-6061cf7112c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:20.373081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa97a4-2044-4aac-9d51-6061cf7112c8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:20.380806 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 230/444] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:20.381622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa97a4-2044-4aac-9d51-6061cf7112c8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:20.422845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525d12f-ab23-4d88-9881-f553883c7fd0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:20.437663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:20.459487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a3a9b-3de8-483e-9e27-5f8d9f7293e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:20.460178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a3a9b-3de8-483e-9e27-5f8d9f7293e3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.460465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a3a9b-3de8-483e-9e27-5f8d9f7293e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.460739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a3a9b-3de8-483e-9e27-5f8d9f7293e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.461581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a3a9b-3de8-483e-9e27-5f8d9f7293e3 None admin] RBAC: Adding query filter params (group.id=1e773e646bd7466eb00fe330aab338f2, role.id=1403cb7e4e264c68b63e467a2e433699, scope.project.id=8c32c58c2df149f4842efa86dea7e18f) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:20.461895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a3a9b-3de8-483e-9e27-5f8d9f7293e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:20.463358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a3a9b-3de8-483e-9e27-5f8d9f7293e3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:20.469518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 215/445] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jul 11 20:44:20 2022] GET /identity/v3/role_assignments?group.id=1e773e646bd7466eb00fe330aab338f2&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=1403cb7e4e264c68b63e467a2e433699 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:20.470123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a3a9b-3de8-483e-9e27-5f8d9f7293e3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:20.488248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa97a4-2044-4aac-9d51-6061cf7112c8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:20.509351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:20.526497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274df61-fbe8-49df-bcb8-020cd1b70e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:20.527361 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274df61-fbe8-49df-bcb8-020cd1b70e91 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.527647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274df61-fbe8-49df-bcb8-020cd1b70e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.527923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274df61-fbe8-49df-bcb8-020cd1b70e91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.541437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274df61-fbe8-49df-bcb8-020cd1b70e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f274df61-fbe8-49df-bcb8-020cd1b70e91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:20.549550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:20.550591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:20.551388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 231/446] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:20.558098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274df61-fbe8-49df-bcb8-020cd1b70e91 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:20.602460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a3a9b-3de8-483e-9e27-5f8d9f7293e3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:20.648703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b160e5-2fbb-4d9a-bdb5-426351689666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:20.649246 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b160e5-2fbb-4d9a-bdb5-426351689666 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.649523 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b160e5-2fbb-4d9a-bdb5-426351689666 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.649789 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b160e5-2fbb-4d9a-bdb5-426351689666 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.674732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274df61-fbe8-49df-bcb8-020cd1b70e91 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:20.691881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:20.716154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee02088-cafd-410c-9472-29101549d718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:20.717092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee02088-cafd-410c-9472-29101549d718 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.717523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee02088-cafd-410c-9472-29101549d718 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.717804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee02088-cafd-410c-9472-29101549d718 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.718575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee02088-cafd-410c-9472-29101549d718 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:20.719274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee02088-cafd-410c-9472-29101549d718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:20.720308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee02088-cafd-410c-9472-29101549d718 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:20.725164 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b160e5-2fbb-4d9a-bdb5-426351689666 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:20.726731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b160e5-2fbb-4d9a-bdb5-426351689666 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:20.730418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 232/447] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:20.736393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee02088-cafd-410c-9472-29101549d718 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:20.766650 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:20.854748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee02088-cafd-410c-9472-29101549d718 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:20.866661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 216/448] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:20.872230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b160e5-2fbb-4d9a-bdb5-426351689666 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:20.893061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79e880-1058-4c79-8016-388e55bbb044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:20.893726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79e880-1058-4c79-8016-388e55bbb044 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:20.894010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79e880-1058-4c79-8016-388e55bbb044 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:20.894299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79e880-1058-4c79-8016-388e55bbb044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:20.942968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:20.965794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79e880-1058-4c79-8016-388e55bbb044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:20.967373 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79e880-1058-4c79-8016-388e55bbb044 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:20.976764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b160e5-2fbb-4d9a-bdb5-426351689666 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:21.005937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:21.011626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2411bf8-4db6-4d26-9042-6b8e240d611b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:21.012526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2411bf8-4db6-4d26-9042-6b8e240d611b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:21.012812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2411bf8-4db6-4d26-9042-6b8e240d611b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:21.013070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2411bf8-4db6-4d26-9042-6b8e240d611b None admin] PATH_INFO: `/v3/projects/8c32c58c2df149f4842efa86dea7e18f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:21.014273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2411bf8-4db6-4d26-9042-6b8e240d611b None admin] RBAC: Authorizing `identity:get_project(project_id=8c32c58c2df149f4842efa86dea7e18f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:21.015730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2411bf8-4db6-4d26-9042-6b8e240d611b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:21.017494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 217/449] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:20 2022] GET /identity/v3/projects/8c32c58c2df149f4842efa86dea7e18f => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:21.023379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2411bf8-4db6-4d26-9042-6b8e240d611b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:21.064835 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 233/450] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:21.070471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79e880-1058-4c79-8016-388e55bbb044 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:21.126833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2411bf8-4db6-4d26-9042-6b8e240d611b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:21.146405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:21.162247 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6e012-b589-4ccf-bbc7-f1e9aad25bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:21.163267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6e012-b589-4ccf-bbc7-f1e9aad25bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:21.163727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6e012-b589-4ccf-bbc7-f1e9aad25bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:21.164030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6e012-b589-4ccf-bbc7-f1e9aad25bd9 None admin] PATH_INFO: `/v3/groups/1e773e646bd7466eb00fe330aab338f2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:21.165294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6e012-b589-4ccf-bbc7-f1e9aad25bd9 None admin] RBAC: Authorizing `identity:get_group(group_id=1e773e646bd7466eb00fe330aab338f2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:21.166775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6e012-b589-4ccf-bbc7-f1e9aad25bd9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:21.168481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 218/451] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:21 2022] GET /identity/v3/groups/1e773e646bd7466eb00fe330aab338f2 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:21.172295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79e880-1058-4c79-8016-388e55bbb044 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:21.174225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6e012-b589-4ccf-bbc7-f1e9aad25bd9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:21.213112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ab5f0-b42c-4641-83cd-1af10923401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:21.213892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ab5f0-b42c-4641-83cd-1af10923401d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:21.214174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ab5f0-b42c-4641-83cd-1af10923401d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:21.214448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ab5f0-b42c-4641-83cd-1af10923401d None admin] PATH_INFO: `/v3/projects/27db080494394cc5b886636221f8edb9` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:21.215662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ab5f0-b42c-4641-83cd-1af10923401d None admin] RBAC: Authorizing `identity:get_project(project_id=27db080494394cc5b886636221f8edb9)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:21.217161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ab5f0-b42c-4641-83cd-1af10923401d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:21.218875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 234/452] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:44:21 2022] GET /identity/v3/projects/27db080494394cc5b886636221f8edb9 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:21.224337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ab5f0-b42c-4641-83cd-1af10923401d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:21.243914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:21.278249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6e012-b589-4ccf-bbc7-f1e9aad25bd9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:21.311585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa852772-92e0-448d-84ef-e9dd603a51d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:21.312264 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa852772-92e0-448d-84ef-e9dd603a51d7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:21.312546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa852772-92e0-448d-84ef-e9dd603a51d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:21.312814 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa852772-92e0-448d-84ef-e9dd603a51d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:21.313649 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa852772-92e0-448d-84ef-e9dd603a51d7 None admin] RBAC: Adding query filter params (group.id=1e773e646bd7466eb00fe330aab338f2, role.id=e2c9006dc77f473f8e1064f285400a98, scope.project.id=8c32c58c2df149f4842efa86dea7e18f) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:21.313960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa852772-92e0-448d-84ef-e9dd603a51d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:21.315388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa852772-92e0-448d-84ef-e9dd603a51d7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:21.321559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 219/453] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jul 11 20:44:21 2022] GET /identity/v3/role_assignments?group.id=1e773e646bd7466eb00fe330aab338f2&scope.project.id=8c32c58c2df149f4842efa86dea7e18f&role.id=e2c9006dc77f473f8e1064f285400a98 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:21.330233 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ab5f0-b42c-4641-83cd-1af10923401d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:21.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b3d43-6621-4918-9f2a-24dd5995df2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:21.369651 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b3d43-6621-4918-9f2a-24dd5995df2a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:21.369934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b3d43-6621-4918-9f2a-24dd5995df2a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:21.370209 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b3d43-6621-4918-9f2a-24dd5995df2a None admin] PATH_INFO: `/v3/groups/8db249922bd6476eb737d3231ac7d44d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:21.371520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b3d43-6621-4918-9f2a-24dd5995df2a None admin] RBAC: Authorizing `identity:get_group(group_id=8db249922bd6476eb737d3231ac7d44d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:21.373049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b3d43-6621-4918-9f2a-24dd5995df2a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:21.374771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 235/454] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jul 11 20:44:21 2022] GET /identity/v3/groups/8db249922bd6476eb737d3231ac7d44d => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:21.380970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa852772-92e0-448d-84ef-e9dd603a51d7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:21.490468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa852772-92e0-448d-84ef-e9dd603a51d7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:21.519717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:21.525683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432987df-c11a-4f35-99f0-deb5c55227c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:21.526361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432987df-c11a-4f35-99f0-deb5c55227c9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:21.526639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432987df-c11a-4f35-99f0-deb5c55227c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:21.526910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432987df-c11a-4f35-99f0-deb5c55227c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:21.527757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432987df-c11a-4f35-99f0-deb5c55227c9 None admin] RBAC: Adding query filter params (group.id=8db249922bd6476eb737d3231ac7d44d, role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=27db080494394cc5b886636221f8edb9) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:21.528068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432987df-c11a-4f35-99f0-deb5c55227c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:21.529551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432987df-c11a-4f35-99f0-deb5c55227c9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:21.535831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 220/455] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jul 11 20:44:21 2022] GET /identity/v3/role_assignments?group.id=8db249922bd6476eb737d3231ac7d44d&scope.project.id=27db080494394cc5b886636221f8edb9&role.id=60bd1c44ce154338a7face8949d07281 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:24.019949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125a3d7-4a80-4b2c-87d9-e06099617ddf None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.020830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125a3d7-4a80-4b2c-87d9-e06099617ddf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.021240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125a3d7-4a80-4b2c-87d9-e06099617ddf None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.023066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 236/456] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:24.028243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3799128-9217-4e42-9646-c8551cdf8467 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.029105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3799128-9217-4e42-9646-c8551cdf8467 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.029588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3799128-9217-4e42-9646-c8551cdf8467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.032576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3f7d4-e17c-4245-bfb4-8882ceb122bb None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.033468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3f7d4-e17c-4245-bfb4-8882ceb122bb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.034468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3f7d4-e17c-4245-bfb4-8882ceb122bb None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.037135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 237/457] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:24.042308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0736-9229-4577-b57a-d76cbd977149 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.042868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0736-9229-4577-b57a-d76cbd977149 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.043331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0736-9229-4577-b57a-d76cbd977149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.059920 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e3799128-9217-4e42-9646-c8551cdf8467 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:24.061152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e3799128-9217-4e42-9646-c8551cdf8467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:24.069190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-770c0736-9229-4577-b57a-d76cbd977149 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:24.070355 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-770c0736-9229-4577-b57a-d76cbd977149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:24.085678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:24.148610 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 221/458] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:24.169481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3799128-9217-4e42-9646-c8551cdf8467 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:24.230701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 238/459] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:24.263051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c0736-9229-4577-b57a-d76cbd977149 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:24.354260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3799128-9217-4e42-9646-c8551cdf8467 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:24.377715 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c0736-9229-4577-b57a-d76cbd977149 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:24.414623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7ad0e-e491-4a11-8910-e2dbc3431928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:24.415974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7ad0e-e491-4a11-8910-e2dbc3431928 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.416947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7ad0e-e491-4a11-8910-e2dbc3431928 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.417669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7ad0e-e491-4a11-8910-e2dbc3431928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.418849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:24.428827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1baa5-95ae-49d9-9062-f5cc77d7632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:24.429361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1baa5-95ae-49d9-9062-f5cc77d7632a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.429646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1baa5-95ae-49d9-9062-f5cc77d7632a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.429919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1baa5-95ae-49d9-9062-f5cc77d7632a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.434077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7ad0e-e491-4a11-8910-e2dbc3431928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:24.435797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7ad0e-e491-4a11-8910-e2dbc3431928 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:24.439610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:24.440613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1baa5-95ae-49d9-9062-f5cc77d7632a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:24.442136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1baa5-95ae-49d9-9062-f5cc77d7632a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c9c7ad0e-e491-4a11-8910-e2dbc3431928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:24.459519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:24.460546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:24.461375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:24.461375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:24.461375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:24.461375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-08c1baa5-95ae-49d9-9062-f5cc77d7632a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:24.461606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:24.462325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:24.463143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:24.463143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:24.463143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:24.463143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:24.463143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 222/460] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:24.468494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41eaf-c405-44e5-8596-72b21e3a7bad None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.468494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41eaf-c405-44e5-8596-72b21e3a7bad None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.468494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41eaf-c405-44e5-8596-72b21e3a7bad None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.469900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 223/461] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:24.470711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 239/462] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:24.471077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41eaf-c405-44e5-8596-72b21e3a7bad None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:24.472762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a46b9-3c00-4ad9-9fc5-a8baebfe3658 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.473232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a46b9-3c00-4ad9-9fc5-a8baebfe3658 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.473667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a46b9-3c00-4ad9-9fc5-a8baebfe3658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.498961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f17a46b9-3c00-4ad9-9fc5-a8baebfe3658 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:24.501204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f17a46b9-3c00-4ad9-9fc5-a8baebfe3658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:24.580246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 240/463] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:24.581325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a46b9-3c00-4ad9-9fc5-a8baebfe3658 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:24.597832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41eaf-c405-44e5-8596-72b21e3a7bad None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:24.638111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdbb7c1-2c60-45e3-b2e4-7917897b7022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:24.638883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbb7c1-2c60-45e3-b2e4-7917897b7022 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.639009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbb7c1-2c60-45e3-b2e4-7917897b7022 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.639233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbb7c1-2c60-45e3-b2e4-7917897b7022 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.639981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdbb7c1-2c60-45e3-b2e4-7917897b7022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:24.640320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdbb7c1-2c60-45e3-b2e4-7917897b7022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:24.641747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdbb7c1-2c60-45e3-b2e4-7917897b7022 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:24.655810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 224/464] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:24.661049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdbb7c1-2c60-45e3-b2e4-7917897b7022 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:24.702151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a46b9-3c00-4ad9-9fc5-a8baebfe3658 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:24.702658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:24.729492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:24.739926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c860e-05ca-40f6-901f-c6c4df175657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:24.741198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c860e-05ca-40f6-901f-c6c4df175657 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.741486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c860e-05ca-40f6-901f-c6c4df175657 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.741764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c860e-05ca-40f6-901f-c6c4df175657 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.742851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c860e-05ca-40f6-901f-c6c4df175657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:24.743165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c860e-05ca-40f6-901f-c6c4df175657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:24.744949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c860e-05ca-40f6-901f-c6c4df175657 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:24.757010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 241/465] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:24.762754 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884485c-3a70-4c6a-83cd-267cd16c035a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.763045 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884485c-3a70-4c6a-83cd-267cd16c035a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.763326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884485c-3a70-4c6a-83cd-267cd16c035a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.764614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 242/466] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:24.766927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdbb7c1-2c60-45e3-b2e4-7917897b7022 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:24.769138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884485c-3a70-4c6a-83cd-267cd16c035a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:24.804262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c89ba7-9b89-42ca-b21f-6cad67b3cdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:24.805503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c89ba7-9b89-42ca-b21f-6cad67b3cdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.806032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c89ba7-9b89-42ca-b21f-6cad67b3cdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.806316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c89ba7-9b89-42ca-b21f-6cad67b3cdb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.819130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c89ba7-9b89-42ca-b21f-6cad67b3cdb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:24.820669 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c89ba7-9b89-42ca-b21f-6cad67b3cdb1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-52c89ba7-9b89-42ca-b21f-6cad67b3cdb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:24.838654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:24.839481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:24.840349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:24.840349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:24.840349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:24.840349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:24.844875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 225/467] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:24.845690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c89ba7-9b89-42ca-b21f-6cad67b3cdb1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:24.885010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884485c-3a70-4c6a-83cd-267cd16c035a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:24.920820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c7437-caa1-411b-b3f2-1bbbbbbb9c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:24.921415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c7437-caa1-411b-b3f2-1bbbbbbb9c8e None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.921690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c7437-caa1-411b-b3f2-1bbbbbbb9c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.922350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c7437-caa1-411b-b3f2-1bbbbbbb9c8e None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.923055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c7437-caa1-411b-b3f2-1bbbbbbb9c8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:24.924875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c7437-caa1-411b-b3f2-1bbbbbbb9c8e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:24.937420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:24.956254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 243/468] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:44:24 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:44:24.956254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a33b-884b-4c11-a245-be382b7bb9e3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:24.956254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a33b-884b-4c11-a245-be382b7bb9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:24.956612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c89ba7-9b89-42ca-b21f-6cad67b3cdb1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:24.956695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a33b-884b-4c11-a245-be382b7bb9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:24.978259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8d35a33b-884b-4c11-a245-be382b7bb9e3 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:24.979489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8d35a33b-884b-4c11-a245-be382b7bb9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:25.040718 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb4b18-f1c6-4ca9-9e58-4e2f9dd81303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:25.041439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb4b18-f1c6-4ca9-9e58-4e2f9dd81303 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:25.041717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb4b18-f1c6-4ca9-9e58-4e2f9dd81303 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:25.041987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb4b18-f1c6-4ca9-9e58-4e2f9dd81303 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:25.042729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb4b18-f1c6-4ca9-9e58-4e2f9dd81303 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:25.044263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb4b18-f1c6-4ca9-9e58-4e2f9dd81303 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:25.056670 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:25.076166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 226/469] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:44:24 2022] POST /identity/v3/users => generated 248 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:44:25.076166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb4b18-f1c6-4ca9-9e58-4e2f9dd81303 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:25.183669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 244/470] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:25.210766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35a33b-884b-4c11-a245-be382b7bb9e3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:25.255058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb4b18-f1c6-4ca9-9e58-4e2f9dd81303 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:25.309881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568e620-3cd2-44d9-aa0f-d7d406ee6eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:25.310577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568e620-3cd2-44d9-aa0f-d7d406ee6eff None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:25.310859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568e620-3cd2-44d9-aa0f-d7d406ee6eff None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:25.311145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568e620-3cd2-44d9-aa0f-d7d406ee6eff None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:25.311873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568e620-3cd2-44d9-aa0f-d7d406ee6eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:25.312206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568e620-3cd2-44d9-aa0f-d7d406ee6eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:25.313595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568e620-3cd2-44d9-aa0f-d7d406ee6eff None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:25.327715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 227/471] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:25.337379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568e620-3cd2-44d9-aa0f-d7d406ee6eff None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:25.351041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:25.380451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35a33b-884b-4c11-a245-be382b7bb9e3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:25.435889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225f03c-27d6-4195-a05d-6b4296ac8ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:25.437113 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225f03c-27d6-4195-a05d-6b4296ac8ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:25.437585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225f03c-27d6-4195-a05d-6b4296ac8ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:25.438048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225f03c-27d6-4195-a05d-6b4296ac8ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:25.468050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568e620-3cd2-44d9-aa0f-d7d406ee6eff None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:25.470564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225f03c-27d6-4195-a05d-6b4296ac8ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:25.474861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225f03c-27d6-4195-a05d-6b4296ac8ec0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:25.508043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26599a86-fa52-43c1-9ae6-a2e7b68dafa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:25.508821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26599a86-fa52-43c1-9ae6-a2e7b68dafa7 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:25.509101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26599a86-fa52-43c1-9ae6-a2e7b68dafa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:25.509373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26599a86-fa52-43c1-9ae6-a2e7b68dafa7 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:25.510092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26599a86-fa52-43c1-9ae6-a2e7b68dafa7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:25.511531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26599a86-fa52-43c1-9ae6-a2e7b68dafa7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f225f03c-27d6-4195-a05d-6b4296ac8ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:25.513453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:25.514274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:25.515211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:25.515211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:25.515211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:25.515211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:25.524634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 245/472] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:25.526488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225f03c-27d6-4195-a05d-6b4296ac8ec0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:25.563063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 228/473] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:44:25 2022] POST /identity/v3/users => generated 248 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:44:25.601035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:25.701133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225f03c-27d6-4195-a05d-6b4296ac8ec0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:25.723299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f8653-9d0e-43b6-8760-036f59e68215 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:25.723682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f8653-9d0e-43b6-8760-036f59e68215 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:25.723860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f8653-9d0e-43b6-8760-036f59e68215 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:25.726222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 229/474] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:44:25.731777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9bf33-faa6-4926-95c4-a83c2970bd21 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:25.732077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9bf33-faa6-4926-95c4-a83c2970bd21 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:25.732379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9bf33-faa6-4926-95c4-a83c2970bd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:25.741749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41c333-8d6c-411c-8a22-ef473e46ae0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:25.742791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41c333-8d6c-411c-8a22-ef473e46ae0e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:25.743317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41c333-8d6c-411c-8a22-ef473e46ae0e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:25.743806 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41c333-8d6c-411c-8a22-ef473e46ae0e None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:25.744828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41c333-8d6c-411c-8a22-ef473e46ae0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:25.745439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41c333-8d6c-411c-8a22-ef473e46ae0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:25.747229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41c333-8d6c-411c-8a22-ef473e46ae0e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:25.760412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 246/475] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:25.766363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41c333-8d6c-411c-8a22-ef473e46ae0e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:25.777350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7bd9bf33-faa6-4926-95c4-a83c2970bd21 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:25.778587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7bd9bf33-faa6-4926-95c4-a83c2970bd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:25.860545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:25.868658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 230/476] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:25.905025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9bf33-faa6-4926-95c4-a83c2970bd21 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:25.948049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:25.968971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41c333-8d6c-411c-8a22-ef473e46ae0e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:26.016737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4a9b8-e96e-4a11-b304-684bb913d8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:26.017443 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4a9b8-e96e-4a11-b304-684bb913d8d5 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:26.018080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4a9b8-e96e-4a11-b304-684bb913d8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:26.018359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4a9b8-e96e-4a11-b304-684bb913d8d5 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:26.019348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4a9b8-e96e-4a11-b304-684bb913d8d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:26.021015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4a9b8-e96e-4a11-b304-684bb913d8d5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:26.043143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9bf33-faa6-4926-95c4-a83c2970bd21 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:26.055768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 247/477] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:44:25 2022] POST /identity/v3/users => generated 249 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:44:26.090013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd1e10-6cc4-43e2-846f-b8f30dcc804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:26.090992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd1e10-6cc4-43e2-846f-b8f30dcc804d None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:26.091409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd1e10-6cc4-43e2-846f-b8f30dcc804d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:26.091833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd1e10-6cc4-43e2-846f-b8f30dcc804d None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:26.092783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd1e10-6cc4-43e2-846f-b8f30dcc804d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:26.094460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd1e10-6cc4-43e2-846f-b8f30dcc804d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:26.102865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.sql.core [None req-88cd1e10-6cc4-43e2-846f-b8f30dcc804d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 20:44:26.102865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 11 20:44:26.102865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [parameters: {'id': '4c3d2d7b5f2b49df981497b5d1be6055', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 11 20:44:26.102865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80310) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-88cd1e10-6cc4-43e2-846f-b8f30dcc804d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application conn.query(q) Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result.read() Jul 11 20:44:26.122268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return ref.to_dict() Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 20:44:26.123414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application next(self.gen) Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application yield resource Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application next(self.gen) Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise self.value Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application session.commit() Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 20:44:26.124525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.session.flush() Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application compat.raise_( Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 20:44:26.125576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 20:44:26.126661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application conn.query(q) Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result.read() Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 20:44:26.127462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [parameters: {'id': '4c3d2d7b5f2b49df981497b5d1be6055', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:26.128272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 11 20:44:26.129057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:26.131868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 231/478] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Jul 11 20:44:25 2022] POST /identity/v3/roles => generated 144 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:26.137508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4a9b8-e96e-4a11-b304-684bb913d8d5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:26.252647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4a9b8-e96e-4a11-b304-684bb913d8d5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:26.295740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47488f49-4b8e-4731-ab07-00981555e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:26.296766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47488f49-4b8e-4731-ab07-00981555e399 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:26.297143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47488f49-4b8e-4731-ab07-00981555e399 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:26.297510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47488f49-4b8e-4731-ab07-00981555e399 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:26.309928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:26.316962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47488f49-4b8e-4731-ab07-00981555e399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-47488f49-4b8e-4731-ab07-00981555e399 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:26.327855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:26.328894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 20:44:26.329841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:26.331928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 248/479] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:44:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:26.338727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd1e10-6cc4-43e2-846f-b8f30dcc804d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:26.403034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:26.450378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd1e10-6cc4-43e2-846f-b8f30dcc804d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:26.485729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931e460-ac04-4647-a5f4-226e056206fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:26.486654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e460-ac04-4647-a5f4-226e056206fc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:26.487079 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e460-ac04-4647-a5f4-226e056206fc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:26.487491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e460-ac04-4647-a5f4-226e056206fc None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:26.488499 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931e460-ac04-4647-a5f4-226e056206fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:26.488944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931e460-ac04-4647-a5f4-226e056206fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:26.490607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931e460-ac04-4647-a5f4-226e056206fc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:26.500655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 232/480] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:27.558839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4243b-5fdd-468e-adbb-f3ef243d14a6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:27.559403 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4243b-5fdd-468e-adbb-f3ef243d14a6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:27.559403 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4243b-5fdd-468e-adbb-f3ef243d14a6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:27.561011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 249/481] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:27.571313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b3a01-dc61-4b1c-a723-15fffb30a3a3 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:27.571774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b3a01-dc61-4b1c-a723-15fffb30a3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:27.571885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b3a01-dc61-4b1c-a723-15fffb30a3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:27.593086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-564b3a01-dc61-4b1c-a723-15fffb30a3a3 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:27.594271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-564b3a01-dc61-4b1c-a723-15fffb30a3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:27.678193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 233/482] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:27.716149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4243b-5fdd-468e-adbb-f3ef243d14a6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:27.732879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2081a-3f8b-414c-92a0-2d9e754a84a3 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:27.733494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2081a-3f8b-414c-92a0-2d9e754a84a3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:27.734255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2081a-3f8b-414c-92a0-2d9e754a84a3 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:27.736292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 234/483] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:44:27.742536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a3920-52e1-4ddc-9ea0-6b914a789e70 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:27.743342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a3920-52e1-4ddc-9ea0-6b914a789e70 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:27.743813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a3920-52e1-4ddc-9ea0-6b914a789e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:27.765125 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-282a3920-52e1-4ddc-9ea0-6b914a789e70 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:27.770204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-282a3920-52e1-4ddc-9ea0-6b914a789e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:27.864217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:27.912570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 235/484] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:27.912781 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4243b-5fdd-468e-adbb-f3ef243d14a6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:27.952088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8aabd-1ecb-43a7-84a7-5c8e27a368ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:27.952944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8aabd-1ecb-43a7-84a7-5c8e27a368ca None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:27.953232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8aabd-1ecb-43a7-84a7-5c8e27a368ca None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:27.953513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8aabd-1ecb-43a7-84a7-5c8e27a368ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:27.964349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8aabd-1ecb-43a7-84a7-5c8e27a368ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-56e8aabd-1ecb-43a7-84a7-5c8e27a368ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:27.972151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:27.973230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:27.974211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:27.976954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a3920-52e1-4ddc-9ea0-6b914a789e70 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:27.987038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 250/485] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:27.992184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8aabd-1ecb-43a7-84a7-5c8e27a368ca None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:28.080567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:28.124159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8aabd-1ecb-43a7-84a7-5c8e27a368ca None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:28.161464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922f340-bc52-45d7-a481-ea51d0481d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:28.162353 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922f340-bc52-45d7-a481-ea51d0481d66 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.162758 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922f340-bc52-45d7-a481-ea51d0481d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.163223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922f340-bc52-45d7-a481-ea51d0481d66 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.163996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922f340-bc52-45d7-a481-ea51d0481d66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:28.164594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922f340-bc52-45d7-a481-ea51d0481d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:28.166146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922f340-bc52-45d7-a481-ea51d0481d66 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:28.175555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 251/486] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:28.181428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922f340-bc52-45d7-a481-ea51d0481d66 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:28.245974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a3920-52e1-4ddc-9ea0-6b914a789e70 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:28.292674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922f340-bc52-45d7-a481-ea51d0481d66 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:28.305948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89548b-1c52-4803-8fd2-8bfb726f908f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:28.307287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89548b-1c52-4803-8fd2-8bfb726f908f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.307816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89548b-1c52-4803-8fd2-8bfb726f908f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.309233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89548b-1c52-4803-8fd2-8bfb726f908f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.322012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:28.325417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89548b-1c52-4803-8fd2-8bfb726f908f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-8f89548b-1c52-4803-8fd2-8bfb726f908f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:28.334796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:28.335711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:28.336512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:28.337125 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42214a2-0d88-4d29-be45-e899d21fb68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:28.337720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42214a2-0d88-4d29-be45-e899d21fb68a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.337720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42214a2-0d88-4d29-be45-e899d21fb68a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.337863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42214a2-0d88-4d29-be45-e899d21fb68a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.339802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 236/487] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:28.341519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea8bc2-0b86-4275-83aa-aecfa5655e68 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.342002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea8bc2-0b86-4275-83aa-aecfa5655e68 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.342464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea8bc2-0b86-4275-83aa-aecfa5655e68 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.347982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 237/488] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:28.349091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea8bc2-0b86-4275-83aa-aecfa5655e68 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:28.349213 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42214a2-0d88-4d29-be45-e899d21fb68a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:28.350985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42214a2-0d88-4d29-be45-e899d21fb68a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-e42214a2-0d88-4d29-be45-e899d21fb68a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:28.369179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:28.370058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:28.371495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:28.371495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:28.371495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:28.371495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:28.371495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 252/489] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:28.371958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5bebb-f6f4-4583-a308-3d8f350d717c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.372267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5bebb-f6f4-4583-a308-3d8f350d717c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.372569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5bebb-f6f4-4583-a308-3d8f350d717c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.391164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-42f5bebb-f6f4-4583-a308-3d8f350d717c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:28.392369 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-42f5bebb-f6f4-4583-a308-3d8f350d717c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:28.398255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:28.467178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea8bc2-0b86-4275-83aa-aecfa5655e68 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:28.484134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 253/490] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:28.484134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5bebb-f6f4-4583-a308-3d8f350d717c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:28.530126 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235cdd7-fcb4-4bd2-996a-36f44b557a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:28.531423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235cdd7-fcb4-4bd2-996a-36f44b557a1c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.532277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235cdd7-fcb4-4bd2-996a-36f44b557a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.533005 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235cdd7-fcb4-4bd2-996a-36f44b557a1c None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.534850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235cdd7-fcb4-4bd2-996a-36f44b557a1c None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:28.535807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235cdd7-fcb4-4bd2-996a-36f44b557a1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:28.537586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235cdd7-fcb4-4bd2-996a-36f44b557a1c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:28.548571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 238/491] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:28.550005 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235cdd7-fcb4-4bd2-996a-36f44b557a1c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:28.605821 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5bebb-f6f4-4583-a308-3d8f350d717c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:28.608299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:28.646771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f213105-02b8-4f07-9adb-28437f7afa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:28.647944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f213105-02b8-4f07-9adb-28437f7afa34 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.648410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f213105-02b8-4f07-9adb-28437f7afa34 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.648815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f213105-02b8-4f07-9adb-28437f7afa34 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.649897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f213105-02b8-4f07-9adb-28437f7afa34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:28.650216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f213105-02b8-4f07-9adb-28437f7afa34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:28.652740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f213105-02b8-4f07-9adb-28437f7afa34 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:28.663781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:28.668198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235cdd7-fcb4-4bd2-996a-36f44b557a1c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:28.669132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 254/492] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:28.670011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f213105-02b8-4f07-9adb-28437f7afa34 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:28.707633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972ff7a-5437-41c1-9a84-1538978b7af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:28.708626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972ff7a-5437-41c1-9a84-1538978b7af0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.709856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972ff7a-5437-41c1-9a84-1538978b7af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.710141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972ff7a-5437-41c1-9a84-1538978b7af0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.722195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972ff7a-5437-41c1-9a84-1538978b7af0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-8972ff7a-5437-41c1-9a84-1538978b7af0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:28.730447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:28.731399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:28.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:28.736749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 239/493] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:28.737584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972ff7a-5437-41c1-9a84-1538978b7af0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:28.800667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f213105-02b8-4f07-9adb-28437f7afa34 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:28.843343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9991f-5536-4da8-bb03-ed22298bc0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:28.844209 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9991f-5536-4da8-bb03-ed22298bc0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.844497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9991f-5536-4da8-bb03-ed22298bc0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.844767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9991f-5536-4da8-bb03-ed22298bc0c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.856940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9991f-5536-4da8-bb03-ed22298bc0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:28.859286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9991f-5536-4da8-bb03-ed22298bc0c4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:28.872143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972ff7a-5437-41c1-9a84-1538978b7af0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d8a9991f-5536-4da8-bb03-ed22298bc0c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:28.880149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:28.881141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:28.881943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:28.881943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:28.881943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:28.881943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:28.882159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 255/494] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:28.883387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6de55-bf7a-4ec0-8d56-8bcd52be4c2c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.883795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6de55-bf7a-4ec0-8d56-8bcd52be4c2c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.884116 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6de55-bf7a-4ec0-8d56-8bcd52be4c2c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.888984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 256/495] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:28.890437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6de55-bf7a-4ec0-8d56-8bcd52be4c2c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:28.912477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:28.917525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59b30e-d32d-4565-badf-fe8b0fb1fd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:28.918350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59b30e-d32d-4565-badf-fe8b0fb1fd4a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:28.918636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59b30e-d32d-4565-badf-fe8b0fb1fd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:28.918919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59b30e-d32d-4565-badf-fe8b0fb1fd4a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:28.931612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59b30e-d32d-4565-badf-fe8b0fb1fd4a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:28.934186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59b30e-d32d-4565-badf-fe8b0fb1fd4a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-bb59b30e-d32d-4565-badf-fe8b0fb1fd4a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:28.954681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:28.955692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:28.956604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 240/496] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:28.957151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59b30e-d32d-4565-badf-fe8b0fb1fd4a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:28.980632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:29.030656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6de55-bf7a-4ec0-8d56-8bcd52be4c2c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:29.082813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22c29e-03d8-4378-8ab5-4288ddd1f67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:29.083589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22c29e-03d8-4378-8ab5-4288ddd1f67d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.084241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22c29e-03d8-4378-8ab5-4288ddd1f67d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.084241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22c29e-03d8-4378-8ab5-4288ddd1f67d None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.085048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22c29e-03d8-4378-8ab5-4288ddd1f67d None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:29.085349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22c29e-03d8-4378-8ab5-4288ddd1f67d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:29.086725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22c29e-03d8-4378-8ab5-4288ddd1f67d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:29.096985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59b30e-d32d-4565-badf-fe8b0fb1fd4a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:29.104942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 257/497] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:29.104942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb94f8-f81f-4dde-9b8c-97997d7b5f93 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.104942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb94f8-f81f-4dde-9b8c-97997d7b5f93 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.104942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb94f8-f81f-4dde-9b8c-97997d7b5f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.121522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-96eb94f8-f81f-4dde-9b8c-97997d7b5f93 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:29.122722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-96eb94f8-f81f-4dde-9b8c-97997d7b5f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:29.136891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45425f6e-0224-441a-8535-9282cf0dc313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:29.137629 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45425f6e-0224-441a-8535-9282cf0dc313 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.137899 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45425f6e-0224-441a-8535-9282cf0dc313 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.138175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45425f6e-0224-441a-8535-9282cf0dc313 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.139022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45425f6e-0224-441a-8535-9282cf0dc313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:29.139335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45425f6e-0224-441a-8535-9282cf0dc313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:29.140893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45425f6e-0224-441a-8535-9282cf0dc313 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:29.158663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 241/498] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:29.158663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45425f6e-0224-441a-8535-9282cf0dc313 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:29.220519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 258/499] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:29.220519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb94f8-f81f-4dde-9b8c-97997d7b5f93 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:29.237560 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:29.254981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:29.274740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45425f6e-0224-441a-8535-9282cf0dc313 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:29.314052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48ddc6-8890-4c71-b7b2-e2db5bffece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:29.314052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48ddc6-8890-4c71-b7b2-e2db5bffece0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.314387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48ddc6-8890-4c71-b7b2-e2db5bffece0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.314563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48ddc6-8890-4c71-b7b2-e2db5bffece0 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.315435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48ddc6-8890-4c71-b7b2-e2db5bffece0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:29.315728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48ddc6-8890-4c71-b7b2-e2db5bffece0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:29.317323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48ddc6-8890-4c71-b7b2-e2db5bffece0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:29.338731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb94f8-f81f-4dde-9b8c-97997d7b5f93 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:29.358867 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 242/500] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jul 11 20:44:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:29.359540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48ddc6-8890-4c71-b7b2-e2db5bffece0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:29.375397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75311104-4685-42bd-9743-91145fa9f3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:29.376235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311104-4685-42bd-9743-91145fa9f3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.377368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311104-4685-42bd-9743-91145fa9f3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.377368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311104-4685-42bd-9743-91145fa9f3a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.388588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75311104-4685-42bd-9743-91145fa9f3a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:29.390072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75311104-4685-42bd-9743-91145fa9f3a0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-75311104-4685-42bd-9743-91145fa9f3a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:29.412322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:29.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:29.414430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:29.414430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:29.414430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:29.414430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:29.416480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 259/501] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:29.417805 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df9e2f-ad0b-4984-8ba2-f6fc6ad2701c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.418418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df9e2f-ad0b-4984-8ba2-f6fc6ad2701c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.418592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df9e2f-ad0b-4984-8ba2-f6fc6ad2701c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.419940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 260/502] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:29.420642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df9e2f-ad0b-4984-8ba2-f6fc6ad2701c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:29.469113 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48ddc6-8890-4c71-b7b2-e2db5bffece0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:29.487894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:29.505913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4dd7d-bcce-46af-85a9-f472a1443516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:29.506726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dd7d-bcce-46af-85a9-f472a1443516 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.507010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dd7d-bcce-46af-85a9-f472a1443516 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.507286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dd7d-bcce-46af-85a9-f472a1443516 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.515141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4dd7d-bcce-46af-85a9-f472a1443516 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:29.516677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4dd7d-bcce-46af-85a9-f472a1443516 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:29.518730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-45f4dd7d-bcce-46af-85a9-f472a1443516 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:29.534304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:29.535409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:29.536442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:29.536442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:29.536442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:29.536442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:29.536442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:29.536442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:29.536442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:29.536442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:29.536442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:44:29.536442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:29.537023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df9e2f-ad0b-4984-8ba2-f6fc6ad2701c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:29.537145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 243/503] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jul 11 20:44:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:29.537145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4dd7d-bcce-46af-85a9-f472a1443516 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:29.576437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720adc-831a-4d51-a576-8da50ce6f1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:29.577237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720adc-831a-4d51-a576-8da50ce6f1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.577516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720adc-831a-4d51-a576-8da50ce6f1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.577786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720adc-831a-4d51-a576-8da50ce6f1a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.588460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e720adc-831a-4d51-a576-8da50ce6f1a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9e720adc-831a-4d51-a576-8da50ce6f1a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:29.596061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:29.597123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:29.597927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 261/504] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:29.598568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720adc-831a-4d51-a576-8da50ce6f1a1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:29.646030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4dd7d-bcce-46af-85a9-f472a1443516 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:29.681641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8633898-934a-4bd1-8f17-34ea8e97459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:29.682345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8633898-934a-4bd1-8f17-34ea8e97459d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.683165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8633898-934a-4bd1-8f17-34ea8e97459d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.683438 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8633898-934a-4bd1-8f17-34ea8e97459d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.699969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720adc-831a-4d51-a576-8da50ce6f1a1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:29.720938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:29.734947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afad156-0cc6-4d62-a9f8-1994ecc99370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:29.735618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afad156-0cc6-4d62-a9f8-1994ecc99370 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.735897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afad156-0cc6-4d62-a9f8-1994ecc99370 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.736202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afad156-0cc6-4d62-a9f8-1994ecc99370 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.736926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afad156-0cc6-4d62-a9f8-1994ecc99370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:29.737230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afad156-0cc6-4d62-a9f8-1994ecc99370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:29.738675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afad156-0cc6-4d62-a9f8-1994ecc99370 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:29.750338 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 262/505] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:29.751732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657c60f-2281-4590-886d-3fbf8fe5abc5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.752024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657c60f-2281-4590-886d-3fbf8fe5abc5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.752332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657c60f-2281-4590-886d-3fbf8fe5abc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.760728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:29.765502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8633898-934a-4bd1-8f17-34ea8e97459d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:29.767007 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8633898-934a-4bd1-8f17-34ea8e97459d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:29.769751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a657c60f-2281-4590-886d-3fbf8fe5abc5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:29.770940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a657c60f-2281-4590-886d-3fbf8fe5abc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:29.838199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 263/506] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:29.839059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657c60f-2281-4590-886d-3fbf8fe5abc5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:29.880398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 244/507] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:29.880398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8633898-934a-4bd1-8f17-34ea8e97459d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:29.941044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657c60f-2281-4590-886d-3fbf8fe5abc5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:29.952317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:29.972219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:29.976735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8633898-934a-4bd1-8f17-34ea8e97459d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:29.977651 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371ec7f-2839-4bf9-a73f-7f306251625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:29.978342 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371ec7f-2839-4bf9-a73f-7f306251625d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:29.978625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371ec7f-2839-4bf9-a73f-7f306251625d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:29.978896 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371ec7f-2839-4bf9-a73f-7f306251625d None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:29.979768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371ec7f-2839-4bf9-a73f-7f306251625d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:29.980061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371ec7f-2839-4bf9-a73f-7f306251625d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:29.981547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371ec7f-2839-4bf9-a73f-7f306251625d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.012352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144efc5b-c4f5-49e8-bd21-275b8a59d2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.012939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144efc5b-c4f5-49e8-bd21-275b8a59d2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.013220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144efc5b-c4f5-49e8-bd21-275b8a59d2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.013472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144efc5b-c4f5-49e8-bd21-275b8a59d2a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.014225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144efc5b-c4f5-49e8-bd21-275b8a59d2a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:30.014526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144efc5b-c4f5-49e8-bd21-275b8a59d2a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.015894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144efc5b-c4f5-49e8-bd21-275b8a59d2a7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.018773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 264/508] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jul 11 20:44:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:30.024273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371ec7f-2839-4bf9-a73f-7f306251625d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.024371 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 245/509] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:30.024932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144efc5b-c4f5-49e8-bd21-275b8a59d2a7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.120475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371ec7f-2839-4bf9-a73f-7f306251625d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.126857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144efc5b-c4f5-49e8-bd21-275b8a59d2a7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.154428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371762b-b63b-4810-bfd8-1c106e455eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.155252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371762b-b63b-4810-bfd8-1c106e455eec None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.155529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371762b-b63b-4810-bfd8-1c106e455eec None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.155798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371762b-b63b-4810-bfd8-1c106e455eec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.161037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513342c5-671b-4524-b1b6-f4e3255cf8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.161836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513342c5-671b-4524-b1b6-f4e3255cf8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.162383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513342c5-671b-4524-b1b6-f4e3255cf8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.162832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513342c5-671b-4524-b1b6-f4e3255cf8b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.163046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371762b-b63b-4810-bfd8-1c106e455eec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.164522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371762b-b63b-4810-bfd8-1c106e455eec None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.173374 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513342c5-671b-4524-b1b6-f4e3255cf8b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.174826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513342c5-671b-4524-b1b6-f4e3255cf8b6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-3371762b-b63b-4810-bfd8-1c106e455eec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:30.180022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:30.180938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:30.181861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 265/510] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:30.182579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371762b-b63b-4810-bfd8-1c106e455eec None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-513342c5-671b-4524-b1b6-f4e3255cf8b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:30.194729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:30.195766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:30.196670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:30.196670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:30.196670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:30.196670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:30.196670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 246/511] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:30.196934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513342c5-671b-4524-b1b6-f4e3255cf8b6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.212586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:30.213191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:30.283773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371762b-b63b-4810-bfd8-1c106e455eec None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.300711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513342c5-671b-4524-b1b6-f4e3255cf8b6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.318427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c1974-f7e8-4e81-b836-7709b2a50304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.319220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c1974-f7e8-4e81-b836-7709b2a50304 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.319489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c1974-f7e8-4e81-b836-7709b2a50304 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.319755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c1974-f7e8-4e81-b836-7709b2a50304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.330160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c1974-f7e8-4e81-b836-7709b2a50304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.331617 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c1974-f7e8-4e81-b836-7709b2a50304 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.335556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7a6e2-a2c1-4a51-8bd8-0fe52e3e70be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.336398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7a6e2-a2c1-4a51-8bd8-0fe52e3e70be None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.336679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7a6e2-a2c1-4a51-8bd8-0fe52e3e70be None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.336960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7a6e2-a2c1-4a51-8bd8-0fe52e3e70be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-502c1974-f7e8-4e81-b836-7709b2a50304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:30.349941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:30.350807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:30.351665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:30.351665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:30.351665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:30.351665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:30.351665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 266/512] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:30.351925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c1974-f7e8-4e81-b836-7709b2a50304 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.408451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7a6e2-a2c1-4a51-8bd8-0fe52e3e70be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.409921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7a6e2-a2c1-4a51-8bd8-0fe52e3e70be None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.423608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:30.434185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:30.454347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c1974-f7e8-4e81-b836-7709b2a50304 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.489935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a6fd0-4f8a-4178-bf18-a924a4cb307a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.490849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a6fd0-4f8a-4178-bf18-a924a4cb307a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.491134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a6fd0-4f8a-4178-bf18-a924a4cb307a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.491412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a6fd0-4f8a-4178-bf18-a924a4cb307a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.501778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a6fd0-4f8a-4178-bf18-a924a4cb307a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.503323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a6fd0-4f8a-4178-bf18-a924a4cb307a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.511374 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 247/513] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:30.512762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7a6e2-a2c1-4a51-8bd8-0fe52e3e70be None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b97a6fd0-4f8a-4178-bf18-a924a4cb307a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:30.521229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:30.522268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:30.523079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:30.523079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:30.523079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:30.523079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:30.523079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 267/514] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:30.523329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a6fd0-4f8a-4178-bf18-a924a4cb307a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.615179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7a6e2-a2c1-4a51-8bd8-0fe52e3e70be None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.623353 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a6fd0-4f8a-4178-bf18-a924a4cb307a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.627789 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:30.649210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a99a-1f64-4477-9ee2-fe44d5b50eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.649880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a99a-1f64-4477-9ee2-fe44d5b50eda None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.650159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a99a-1f64-4477-9ee2-fe44d5b50eda None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.650422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a99a-1f64-4477-9ee2-fe44d5b50eda None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.651270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a99a-1f64-4477-9ee2-fe44d5b50eda None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:30.651574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a99a-1f64-4477-9ee2-fe44d5b50eda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.653030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a99a-1f64-4477-9ee2-fe44d5b50eda None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.653153 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:30.658165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0a4c5-d62a-4021-8918-03b8e2a58b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.658829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0a4c5-d62a-4021-8918-03b8e2a58b4c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.659109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0a4c5-d62a-4021-8918-03b8e2a58b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.659383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0a4c5-d62a-4021-8918-03b8e2a58b4c None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.660149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0a4c5-d62a-4021-8918-03b8e2a58b4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:30.660443 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0a4c5-d62a-4021-8918-03b8e2a58b4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.661797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0a4c5-d62a-4021-8918-03b8e2a58b4c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.670833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 248/515] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:30.675545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a99a-1f64-4477-9ee2-fe44d5b50eda None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.675638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 268/516] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:30.675638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0a4c5-d62a-4021-8918-03b8e2a58b4c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.770882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a99a-1f64-4477-9ee2-fe44d5b50eda None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.772316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0a4c5-d62a-4021-8918-03b8e2a58b4c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.805524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3ce29-e18e-4bfb-bd3b-821fdeb6ceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.805992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5cca02-0402-464a-9ee6-ffb3eaafed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.806317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3ce29-e18e-4bfb-bd3b-821fdeb6ceb4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.806593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3ce29-e18e-4bfb-bd3b-821fdeb6ceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.806688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cca02-0402-464a-9ee6-ffb3eaafed4b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.806859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3ce29-e18e-4bfb-bd3b-821fdeb6ceb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.806952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cca02-0402-464a-9ee6-ffb3eaafed4b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.807182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cca02-0402-464a-9ee6-ffb3eaafed4b None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.807916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cca02-0402-464a-9ee6-ffb3eaafed4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:30.808261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cca02-0402-464a-9ee6-ffb3eaafed4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.809648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cca02-0402-464a-9ee6-ffb3eaafed4b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.817424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3ce29-e18e-4bfb-bd3b-821fdeb6ceb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.818947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3ce29-e18e-4bfb-bd3b-821fdeb6ceb4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.825245 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 249/517] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:30.826566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5cca02-0402-464a-9ee6-ffb3eaafed4b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-87b3ce29-e18e-4bfb-bd3b-821fdeb6ceb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:30.836759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:30.837657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:30.838466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:30.838466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:30.838466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:30.838466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:30.838466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 269/518] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:30.838737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3ce29-e18e-4bfb-bd3b-821fdeb6ceb4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:30.858388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:30.887366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:30.926818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5cca02-0402-464a-9ee6-ffb3eaafed4b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.942954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3ce29-e18e-4bfb-bd3b-821fdeb6ceb4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:30.961128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eb1f9-d244-452e-a802-63c1c7d69528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.961799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eb1f9-d244-452e-a802-63c1c7d69528 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.962405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eb1f9-d244-452e-a802-63c1c7d69528 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.962405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eb1f9-d244-452e-a802-63c1c7d69528 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.963069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245eb1f9-d244-452e-a802-63c1c7d69528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:30.963376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245eb1f9-d244-452e-a802-63c1c7d69528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:30.964803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245eb1f9-d244-452e-a802-63c1c7d69528 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:30.975659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d876e28-43be-41df-8d8a-0a34c2ab66bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:30.976336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d876e28-43be-41df-8d8a-0a34c2ab66bf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:30.976613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d876e28-43be-41df-8d8a-0a34c2ab66bf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:30.976888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d876e28-43be-41df-8d8a-0a34c2ab66bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:30.976981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 250/519] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:30.977785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eb1f9-d244-452e-a802-63c1c7d69528 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.061883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d876e28-43be-41df-8d8a-0a34c2ab66bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:31.063377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d876e28-43be-41df-8d8a-0a34c2ab66bf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:31.081684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:31.091098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eb1f9-d244-452e-a802-63c1c7d69528 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:31.108004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:31.126544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a2614-c362-4979-8dfe-80ba1e585672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:31.127233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a2614-c362-4979-8dfe-80ba1e585672 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:31.127513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a2614-c362-4979-8dfe-80ba1e585672 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:31.127791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a2614-c362-4979-8dfe-80ba1e585672 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:31.128577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a2614-c362-4979-8dfe-80ba1e585672 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:31.130024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a2614-c362-4979-8dfe-80ba1e585672 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:31.149734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 251/520] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:44:30 2022] POST /identity/v3/users => generated 247 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:44:31.150476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a2614-c362-4979-8dfe-80ba1e585672 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.166242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 270/521] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:31.167442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d876e28-43be-41df-8d8a-0a34c2ab66bf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.268065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a2614-c362-4979-8dfe-80ba1e585672 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:31.277694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d876e28-43be-41df-8d8a-0a34c2ab66bf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:31.304563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16205cbe-0374-4b2c-adf4-f337bc4dc0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:31.305379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16205cbe-0374-4b2c-adf4-f337bc4dc0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:31.305722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16205cbe-0374-4b2c-adf4-f337bc4dc0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:31.306096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16205cbe-0374-4b2c-adf4-f337bc4dc0b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:31.314065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16205cbe-0374-4b2c-adf4-f337bc4dc0b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:31.315558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16205cbe-0374-4b2c-adf4-f337bc4dc0b8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:31.319847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080be7dd-36fd-47b3-96ca-b752e725948e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:31.320560 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080be7dd-36fd-47b3-96ca-b752e725948e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:31.320846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080be7dd-36fd-47b3-96ca-b752e725948e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:31.321691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080be7dd-36fd-47b3-96ca-b752e725948e None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:31.321876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080be7dd-36fd-47b3-96ca-b752e725948e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:31.322158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080be7dd-36fd-47b3-96ca-b752e725948e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:31.323881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080be7dd-36fd-47b3-96ca-b752e725948e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-16205cbe-0374-4b2c-adf4-f337bc4dc0b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:31.331724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:31.332660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:31.333484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:31.333484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:31.333484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:31.333484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:31.333484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:31.333484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:31.333484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:31.333484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:31.333484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 252/522] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:31.333984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16205cbe-0374-4b2c-adf4-f337bc4dc0b8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.341640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 271/523] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:31.342439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080be7dd-36fd-47b3-96ca-b752e725948e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.343203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:31.346774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:31.461286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16205cbe-0374-4b2c-adf4-f337bc4dc0b8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:31.471197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080be7dd-36fd-47b3-96ca-b752e725948e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:31.495637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c35012-be7a-4402-aafe-8d2b5920ff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:31.496469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c35012-be7a-4402-aafe-8d2b5920ff29 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:31.496796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c35012-be7a-4402-aafe-8d2b5920ff29 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:31.497075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c35012-be7a-4402-aafe-8d2b5920ff29 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:31.504752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c35012-be7a-4402-aafe-8d2b5920ff29 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:31.506350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c35012-be7a-4402-aafe-8d2b5920ff29 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:31.511404 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d3a74-a52a-4c17-b188-892b4360806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:31.512249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d3a74-a52a-4c17-b188-892b4360806d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:31.512534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d3a74-a52a-4c17-b188-892b4360806d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:31.512809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d3a74-a52a-4c17-b188-892b4360806d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-98c35012-be7a-4402-aafe-8d2b5920ff29 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:31.521365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:31.522362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:31.523213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 253/524] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:31.523750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c35012-be7a-4402-aafe-8d2b5920ff29 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.529298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d3a74-a52a-4c17-b188-892b4360806d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:31.530779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d3a74-a52a-4c17-b188-892b4360806d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-132d3a74-a52a-4c17-b188-892b4360806d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:31.551488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:31.552458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:31.558201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:31.558201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:31.558201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:31.558201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:31.558201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 272/525] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:31.558201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d3a74-a52a-4c17-b188-892b4360806d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.595180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:31.622268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:31.637224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c35012-be7a-4402-aafe-8d2b5920ff29 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:31.672359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e835e-4aeb-4a14-b9b0-0155dc9eac2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:31.673068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e835e-4aeb-4a14-b9b0-0155dc9eac2d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:31.673361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e835e-4aeb-4a14-b9b0-0155dc9eac2d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:31.673628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e835e-4aeb-4a14-b9b0-0155dc9eac2d None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:31.674434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e835e-4aeb-4a14-b9b0-0155dc9eac2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:31.674745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e835e-4aeb-4a14-b9b0-0155dc9eac2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:31.676221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e835e-4aeb-4a14-b9b0-0155dc9eac2d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:31.676696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d3a74-a52a-4c17-b188-892b4360806d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:31.699923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 254/526] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:31.699923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e835e-4aeb-4a14-b9b0-0155dc9eac2d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.724200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec698c0c-aba2-4e01-9362-3a7f2f683a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:31.725027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec698c0c-aba2-4e01-9362-3a7f2f683a32 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:31.725310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec698c0c-aba2-4e01-9362-3a7f2f683a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:31.725585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec698c0c-aba2-4e01-9362-3a7f2f683a32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:31.733684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec698c0c-aba2-4e01-9362-3a7f2f683a32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:31.735255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec698c0c-aba2-4e01-9362-3a7f2f683a32 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ec698c0c-aba2-4e01-9362-3a7f2f683a32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:31.750316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:31.751183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:31.752043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:31.752043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:31.752043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:31.752043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:31.752043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:31.752043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:31.752043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:31.752043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:31.752043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 273/527] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:31.753267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec698c0c-aba2-4e01-9362-3a7f2f683a32 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.812747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e835e-4aeb-4a14-b9b0-0155dc9eac2d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:31.844056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:31.854917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc840a-7628-4d9f-ab9b-c2d24152fcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:31.855862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc840a-7628-4d9f-ab9b-c2d24152fcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:31.856167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc840a-7628-4d9f-ab9b-c2d24152fcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:31.856446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc840a-7628-4d9f-ab9b-c2d24152fcf8 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:31.857924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc840a-7628-4d9f-ab9b-c2d24152fcf8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:31.858236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc840a-7628-4d9f-ab9b-c2d24152fcf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:31.860225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc840a-7628-4d9f-ab9b-c2d24152fcf8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:31.866511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec698c0c-aba2-4e01-9362-3a7f2f683a32 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:31.882276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 255/528] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Jul 11 20:44:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:31.883121 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc840a-7628-4d9f-ab9b-c2d24152fcf8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.888784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:31.908417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2eff4-be0b-41cb-aed1-27c198141641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:31.909181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2eff4-be0b-41cb-aed1-27c198141641 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:31.909531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2eff4-be0b-41cb-aed1-27c198141641 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:31.909885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2eff4-be0b-41cb-aed1-27c198141641 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:31.910616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2eff4-be0b-41cb-aed1-27c198141641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:31.911161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2eff4-be0b-41cb-aed1-27c198141641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:31.912368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2eff4-be0b-41cb-aed1-27c198141641 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:31.924779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 274/529] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:31.925414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2eff4-be0b-41cb-aed1-27c198141641 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:31.992404 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc840a-7628-4d9f-ab9b-c2d24152fcf8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:32.026380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942e35d-e76e-4707-9c66-59b92d70dac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:32.027045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942e35d-e76e-4707-9c66-59b92d70dac9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:32.027326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942e35d-e76e-4707-9c66-59b92d70dac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:32.027601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942e35d-e76e-4707-9c66-59b92d70dac9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:32.028482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942e35d-e76e-4707-9c66-59b92d70dac9 None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=097b12fbf6494336b11ece5c498a4103) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:32.028795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942e35d-e76e-4707-9c66-59b92d70dac9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:32.030238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942e35d-e76e-4707-9c66-59b92d70dac9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:32.044143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 256/530] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:31 2022] GET /identity/v3/role_assignments?user.id=097b12fbf6494336b11ece5c498a4103&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:32.044143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942e35d-e76e-4707-9c66-59b92d70dac9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:32.048467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2eff4-be0b-41cb-aed1-27c198141641 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:32.099220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77240b-cb83-4adc-8f9b-875b2089142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:32.100188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77240b-cb83-4adc-8f9b-875b2089142f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:32.100482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77240b-cb83-4adc-8f9b-875b2089142f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:32.100748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77240b-cb83-4adc-8f9b-875b2089142f None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:32.101557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77240b-cb83-4adc-8f9b-875b2089142f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:32.101864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77240b-cb83-4adc-8f9b-875b2089142f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:32.103363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77240b-cb83-4adc-8f9b-875b2089142f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:32.112357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:32.128804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 275/531] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:32.128804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77240b-cb83-4adc-8f9b-875b2089142f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:32.165461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:32.194381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942e35d-e76e-4707-9c66-59b92d70dac9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:32.243630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12cad4-08da-45a1-abd1-3b5375564668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:32.244245 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12cad4-08da-45a1-abd1-3b5375564668 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:32.244534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12cad4-08da-45a1-abd1-3b5375564668 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:32.244811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12cad4-08da-45a1-abd1-3b5375564668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:32.246956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77240b-cb83-4adc-8f9b-875b2089142f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:32.287411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ddf19-f269-4d23-a549-d5fe9b600d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:32.288813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ddf19-f269-4d23-a549-d5fe9b600d69 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:32.289105 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ddf19-f269-4d23-a549-d5fe9b600d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:32.289380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ddf19-f269-4d23-a549-d5fe9b600d69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:32.301047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ddf19-f269-4d23-a549-d5fe9b600d69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:32.303226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ddf19-f269-4d23-a549-d5fe9b600d69 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:32.320236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12cad4-08da-45a1-abd1-3b5375564668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:32.321911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12cad4-08da-45a1-abd1-3b5375564668 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-750ddf19-f269-4d23-a549-d5fe9b600d69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:32.322347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:32.323139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:32.323966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:32.323966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:32.323966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:32.323966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:32.323966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:32.323966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:32.323966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:32.323966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:32.323966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 276/532] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:32.325662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ddf19-f269-4d23-a549-d5fe9b600d69 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:32.376296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:32.434707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 257/533] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:32.435952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12cad4-08da-45a1-abd1-3b5375564668 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:32.456493 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:32.465337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ddf19-f269-4d23-a549-d5fe9b600d69 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:32.513831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd4fc6-5083-4e49-be94-41b72e4ca760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:32.514416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd4fc6-5083-4e49-be94-41b72e4ca760 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:32.514695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd4fc6-5083-4e49-be94-41b72e4ca760 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:32.514967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd4fc6-5083-4e49-be94-41b72e4ca760 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:32.516358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd4fc6-5083-4e49-be94-41b72e4ca760 None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=acb3ad858e4848b9a325c72c7328c863) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:32.516672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd4fc6-5083-4e49-be94-41b72e4ca760 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:32.518599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd4fc6-5083-4e49-be94-41b72e4ca760 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:32.527489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 277/534] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:32 2022] GET /identity/v3/role_assignments?user.id=acb3ad858e4848b9a325c72c7328c863&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:32.528357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd4fc6-5083-4e49-be94-41b72e4ca760 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:32.545831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12cad4-08da-45a1-abd1-3b5375564668 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:32.650439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc3aca-a47f-4d7e-949f-aadeca395d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:32.651311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc3aca-a47f-4d7e-949f-aadeca395d78 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:32.651595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc3aca-a47f-4d7e-949f-aadeca395d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:32.651874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc3aca-a47f-4d7e-949f-aadeca395d78 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:32.652727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc3aca-a47f-4d7e-949f-aadeca395d78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:32.653040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc3aca-a47f-4d7e-949f-aadeca395d78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:32.654521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc3aca-a47f-4d7e-949f-aadeca395d78 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:32.679243 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd4fc6-5083-4e49-be94-41b72e4ca760 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:32.682189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 258/535] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:32.688508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc3aca-a47f-4d7e-949f-aadeca395d78 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:32.706641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:32.749349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a8b91-d296-4ca1-9e04-fe0ded4a3505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:32.750161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8b91-d296-4ca1-9e04-fe0ded4a3505 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:32.750443 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8b91-d296-4ca1-9e04-fe0ded4a3505 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:32.750719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8b91-d296-4ca1-9e04-fe0ded4a3505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:32.773179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a8b91-d296-4ca1-9e04-fe0ded4a3505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:32.774593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a8b91-d296-4ca1-9e04-fe0ded4a3505 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-900a8b91-d296-4ca1-9e04-fe0ded4a3505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:32.801489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:32.802416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:32.803424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:32.803424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:32.803424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:32.803424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:32.803424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 278/536] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:32.815887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a8b91-d296-4ca1-9e04-fe0ded4a3505 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:32.848135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:32.891698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc3aca-a47f-4d7e-949f-aadeca395d78 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:32.956790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb3029-e8f2-4863-9139-6c4d3c7f686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:32.957461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb3029-e8f2-4863-9139-6c4d3c7f686e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:32.957748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb3029-e8f2-4863-9139-6c4d3c7f686e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:32.958026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb3029-e8f2-4863-9139-6c4d3c7f686e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:32.958929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb3029-e8f2-4863-9139-6c4d3c7f686e None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=dee928cd256446cb9e000ceb6e9ef636) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:32.959307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb3029-e8f2-4863-9139-6c4d3c7f686e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:32.960853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb3029-e8f2-4863-9139-6c4d3c7f686e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:32.971717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 259/537] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:32 2022] GET /identity/v3/role_assignments?user.id=dee928cd256446cb9e000ceb6e9ef636&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:32.988307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a8b91-d296-4ca1-9e04-fe0ded4a3505 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:33.030448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbf7d5-f933-4ab5-93e6-47c6d818be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:33.031042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbf7d5-f933-4ab5-93e6-47c6d818be60 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:33.031328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbf7d5-f933-4ab5-93e6-47c6d818be60 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:33.031608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbf7d5-f933-4ab5-93e6-47c6d818be60 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:33.032388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbf7d5-f933-4ab5-93e6-47c6d818be60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:33.032701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbf7d5-f933-4ab5-93e6-47c6d818be60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:33.034096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbf7d5-f933-4ab5-93e6-47c6d818be60 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:33.055639 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 279/538] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:33.061108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb3029-e8f2-4863-9139-6c4d3c7f686e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:33.142989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:33.189930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb3029-e8f2-4863-9139-6c4d3c7f686e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:33.241774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e72f3-8acb-4dfd-8808-e23488c9fa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:33.242603 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e72f3-8acb-4dfd-8808-e23488c9fa99 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:33.242899 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e72f3-8acb-4dfd-8808-e23488c9fa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:33.243163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e72f3-8acb-4dfd-8808-e23488c9fa99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:33.253884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e72f3-8acb-4dfd-8808-e23488c9fa99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:33.255348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e72f3-8acb-4dfd-8808-e23488c9fa99 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-cf3e72f3-8acb-4dfd-8808-e23488c9fa99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:33.276305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:33.278323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:33.280370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:33.280370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:33.280370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:33.280370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:33.280370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:33.280370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:33.280370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:33.280370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:33.280370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 260/539] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:33.287237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbf7d5-f933-4ab5-93e6-47c6d818be60 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:33.385746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af7bd0-806e-4b87-bc6a-29ef432a4f4b None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:33.386072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af7bd0-806e-4b87-bc6a-29ef432a4f4b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:33.386294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af7bd0-806e-4b87-bc6a-29ef432a4f4b None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:33.387754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 261/540] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:33.393166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:33.393808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb614-ad14-4853-832a-00f7260ae931 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:33.394096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb614-ad14-4853-832a-00f7260ae931 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:33.394370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb614-ad14-4853-832a-00f7260ae931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:33.419722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bc5fb614-ad14-4853-832a-00f7260ae931 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:33.419722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bc5fb614-ad14-4853-832a-00f7260ae931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:33.437006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbf7d5-f933-4ab5-93e6-47c6d818be60 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:33.493625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 262/541] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:33.501024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424053b3-5a29-435e-aadc-4cac2df03d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:33.501726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424053b3-5a29-435e-aadc-4cac2df03d0e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:33.502014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424053b3-5a29-435e-aadc-4cac2df03d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:33.502289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424053b3-5a29-435e-aadc-4cac2df03d0e None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:33.503105 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424053b3-5a29-435e-aadc-4cac2df03d0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:33.503412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424053b3-5a29-435e-aadc-4cac2df03d0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:33.504903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424053b3-5a29-435e-aadc-4cac2df03d0e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:33.518213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fb614-ad14-4853-832a-00f7260ae931 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:33.526987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 280/542] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:33.532699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424053b3-5a29-435e-aadc-4cac2df03d0e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:33.584014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:33.655866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fb614-ad14-4853-832a-00f7260ae931 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:33.669796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424053b3-5a29-435e-aadc-4cac2df03d0e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:33.714825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4bf67-7cc7-4e7e-aef5-d21be6cdbdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:33.715641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4bf67-7cc7-4e7e-aef5-d21be6cdbdfb None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:33.715920 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4bf67-7cc7-4e7e-aef5-d21be6cdbdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:33.716212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4bf67-7cc7-4e7e-aef5-d21be6cdbdfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:33.731449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686275d-019e-46b8-b847-c58586ba2788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:33.732128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686275d-019e-46b8-b847-c58586ba2788 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:33.732413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686275d-019e-46b8-b847-c58586ba2788 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:33.732691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686275d-019e-46b8-b847-c58586ba2788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:33.733532 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686275d-019e-46b8-b847-c58586ba2788 None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=0574d5c8974f43218653208ca72c43bb) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:33.733843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686275d-019e-46b8-b847-c58586ba2788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:33.738127 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686275d-019e-46b8-b847-c58586ba2788 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:33.740458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:33.740864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4bf67-7cc7-4e7e-aef5-d21be6cdbdfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:33.752267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 281/543] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:33 2022] GET /identity/v3/role_assignments?user.id=0574d5c8974f43218653208ca72c43bb&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-09f4bf67-7cc7-4e7e-aef5-d21be6cdbdfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:33.753554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:33.754411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:33.755596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 263/544] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:33.761118 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686275d-019e-46b8-b847-c58586ba2788 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:33.907172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686275d-019e-46b8-b847-c58586ba2788 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:33.970009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779c0c5-bd55-422e-abfe-1252a74c15fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:33.971107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779c0c5-bd55-422e-abfe-1252a74c15fd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:33.971605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779c0c5-bd55-422e-abfe-1252a74c15fd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:33.972072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779c0c5-bd55-422e-abfe-1252a74c15fd None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:33.973092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779c0c5-bd55-422e-abfe-1252a74c15fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:33.973620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779c0c5-bd55-422e-abfe-1252a74c15fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:33.975364 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779c0c5-bd55-422e-abfe-1252a74c15fd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:33.985537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 282/545] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:33.992134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4bf67-7cc7-4e7e-aef5-d21be6cdbdfb None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:34.131435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4bf67-7cc7-4e7e-aef5-d21be6cdbdfb None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:34.154723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:34.167996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b785f-4021-449e-bb9b-09d5d5875f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:34.168765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b785f-4021-449e-bb9b-09d5d5875f44 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:34.169045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b785f-4021-449e-bb9b-09d5d5875f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:34.169322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b785f-4021-449e-bb9b-09d5d5875f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:34.180694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b785f-4021-449e-bb9b-09d5d5875f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:34.182155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b785f-4021-449e-bb9b-09d5d5875f44 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-6a0b785f-4021-449e-bb9b-09d5d5875f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:34.200511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:34.201370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:34.202274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:34.202274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:34.202274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:34.202274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:34.202274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 264/546] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:34.207437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779c0c5-bd55-422e-abfe-1252a74c15fd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:34.307450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:34.360274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779c0c5-bd55-422e-abfe-1252a74c15fd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:34.401704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d79e95-ab2c-467c-82de-fc5cce8816b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:34.402375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d79e95-ab2c-467c-82de-fc5cce8816b4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:34.402653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d79e95-ab2c-467c-82de-fc5cce8816b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:34.402921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d79e95-ab2c-467c-82de-fc5cce8816b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:34.403645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d79e95-ab2c-467c-82de-fc5cce8816b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:34.403944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d79e95-ab2c-467c-82de-fc5cce8816b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:34.405351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d79e95-ab2c-467c-82de-fc5cce8816b4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:34.423330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 283/547] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:34.428548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b785f-4021-449e-bb9b-09d5d5875f44 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:34.568417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b785f-4021-449e-bb9b-09d5d5875f44 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:34.618439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3f012-990e-495b-b1de-090f7542b100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:34.619266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3f012-990e-495b-b1de-090f7542b100 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:34.619549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3f012-990e-495b-b1de-090f7542b100 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:34.619825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3f012-990e-495b-b1de-090f7542b100 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:34.631895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc3f012-990e-495b-b1de-090f7542b100 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:34.633425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc3f012-990e-495b-b1de-090f7542b100 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-dbc3f012-990e-495b-b1de-090f7542b100 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:34.650884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:34.651895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:34.653777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 265/548] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:34.657752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d79e95-ab2c-467c-82de-fc5cce8816b4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:34.658859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a3957-36d9-4c79-99fb-588883a0fb74 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:34.659143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a3957-36d9-4c79-99fb-588883a0fb74 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:34.659415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a3957-36d9-4c79-99fb-588883a0fb74 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:34.661090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 266/549] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:44:34.666400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c97325-6115-47a9-a72a-dd11c8ec858a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:34.666679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c97325-6115-47a9-a72a-dd11c8ec858a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:34.666949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c97325-6115-47a9-a72a-dd11c8ec858a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:34.684316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-73c97325-6115-47a9-a72a-dd11c8ec858a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:34.685528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-73c97325-6115-47a9-a72a-dd11c8ec858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:34.755393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 267/550] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:34.756097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:34.779408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c97325-6115-47a9-a72a-dd11c8ec858a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:34.837818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d79e95-ab2c-467c-82de-fc5cce8816b4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:34.874967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:34.901775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac67db-5bd8-47fc-8a53-e21dde35a584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:34.903314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac67db-5bd8-47fc-8a53-e21dde35a584 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:34.906278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac67db-5bd8-47fc-8a53-e21dde35a584 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:34.906696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac67db-5bd8-47fc-8a53-e21dde35a584 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:34.907722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac67db-5bd8-47fc-8a53-e21dde35a584 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:34.908209 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac67db-5bd8-47fc-8a53-e21dde35a584 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:34.911185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac67db-5bd8-47fc-8a53-e21dde35a584 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:34.950038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 284/551] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jul 11 20:44:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:34.957876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac67db-5bd8-47fc-8a53-e21dde35a584 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:34.978613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c97325-6115-47a9-a72a-dd11c8ec858a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:35.046558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f718135-081c-4cbb-8ffa-71884e88e7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:35.048280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f718135-081c-4cbb-8ffa-71884e88e7ab None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.049579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f718135-081c-4cbb-8ffa-71884e88e7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.050764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f718135-081c-4cbb-8ffa-71884e88e7ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.070151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f718135-081c-4cbb-8ffa-71884e88e7ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-7f718135-081c-4cbb-8ffa-71884e88e7ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:35.086505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:35.087700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:35.088837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:35.097019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 268/552] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:35.101530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f718135-081c-4cbb-8ffa-71884e88e7ab None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:35.142122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:35.144575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac67db-5bd8-47fc-8a53-e21dde35a584 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:35.188154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae46ac9-cf9f-4f93-a453-ac215e1805fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:35.188861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae46ac9-cf9f-4f93-a453-ac215e1805fa None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.189147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae46ac9-cf9f-4f93-a453-ac215e1805fa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.189426 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae46ac9-cf9f-4f93-a453-ac215e1805fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.244166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:35.255097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f718135-081c-4cbb-8ffa-71884e88e7ab None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:35.266592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae46ac9-cf9f-4f93-a453-ac215e1805fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:35.268198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae46ac9-cf9f-4f93-a453-ac215e1805fa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:35.291968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315109c-d782-4804-9638-7740ab7189fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:35.292971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315109c-d782-4804-9638-7740ab7189fa None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.293404 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315109c-d782-4804-9638-7740ab7189fa None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.293822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315109c-d782-4804-9638-7740ab7189fa None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.294714 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315109c-d782-4804-9638-7740ab7189fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:35.295174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315109c-d782-4804-9638-7740ab7189fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:35.296728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315109c-d782-4804-9638-7740ab7189fa None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:35.310446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 269/553] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:35.312026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba126-8d1e-4d35-82e3-fbd959556fa2 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.312469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba126-8d1e-4d35-82e3-fbd959556fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.312875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba126-8d1e-4d35-82e3-fbd959556fa2 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.314549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 270/554] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:35.315364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdba126-8d1e-4d35-82e3-fbd959556fa2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:35.383330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 285/555] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:35.384919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de435605-7771-428b-9654-bbca48c5a6f7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.385352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de435605-7771-428b-9654-bbca48c5a6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.385860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de435605-7771-428b-9654-bbca48c5a6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.402434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-de435605-7771-428b-9654-bbca48c5a6f7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:35.403613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-de435605-7771-428b-9654-bbca48c5a6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:35.415239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:35.434238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdba126-8d1e-4d35-82e3-fbd959556fa2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:35.469850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7707c-04a6-4331-8cbc-46adb35e66e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:35.471019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7707c-04a6-4331-8cbc-46adb35e66e9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.471454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7707c-04a6-4331-8cbc-46adb35e66e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.471732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7707c-04a6-4331-8cbc-46adb35e66e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.474498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 286/556] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:35.476345 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b935b-1734-4e61-a6da-ac64c81b8d69 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.476638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b935b-1734-4e61-a6da-ac64c81b8d69 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.476918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b935b-1734-4e61-a6da-ac64c81b8d69 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.478330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 287/557] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:35.483149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b935b-1734-4e61-a6da-ac64c81b8d69 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:35.490433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7707c-04a6-4331-8cbc-46adb35e66e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:35.492379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7707c-04a6-4331-8cbc-46adb35e66e9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:35.496556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d7e7707c-04a6-4331-8cbc-46adb35e66e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:35.524442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:35.525385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:35.526251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:35.526251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:35.526251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:35.526251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:35.531849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 271/558] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:35.533362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a98e-f02e-4d1f-be65-e86e3092dcb9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.533651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a98e-f02e-4d1f-be65-e86e3092dcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.533928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a98e-f02e-4d1f-be65-e86e3092dcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.552441 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9d72a98e-f02e-4d1f-be65-e86e3092dcb9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:35.553977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9d72a98e-f02e-4d1f-be65-e86e3092dcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:35.597346 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b935b-1734-4e61-a6da-ac64c81b8d69 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:35.622156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 272/559] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:35.623254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72a98e-f02e-4d1f-be65-e86e3092dcb9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:35.640970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4942-b8d1-41d2-9c70-c434b86eb3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:35.641839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4942-b8d1-41d2-9c70-c434b86eb3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.642127 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4942-b8d1-41d2-9c70-c434b86eb3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.642449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4942-b8d1-41d2-9c70-c434b86eb3e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.654508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a4942-b8d1-41d2-9c70-c434b86eb3e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:35.655980 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a4942-b8d1-41d2-9c70-c434b86eb3e9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-760a4942-b8d1-41d2-9c70-c434b86eb3e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:35.674375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:35.680612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:35.681717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:35.681717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:35.681717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:35.681717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:35.681717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 288/560] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:35.681717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4942-b8d1-41d2-9c70-c434b86eb3e9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:35.709883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:35.745441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:35.752756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72a98e-f02e-4d1f-be65-e86e3092dcb9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:35.789123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c57248-c6a0-4397-ab34-fdfc291f20d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:35.789933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c57248-c6a0-4397-ab34-fdfc291f20d8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.790158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4942-b8d1-41d2-9c70-c434b86eb3e9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:35.790259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c57248-c6a0-4397-ab34-fdfc291f20d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.790818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c57248-c6a0-4397-ab34-fdfc291f20d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.803721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c57248-c6a0-4397-ab34-fdfc291f20d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e6c57248-c6a0-4397-ab34-fdfc291f20d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:35.811366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:35.812315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:35.813221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 273/561] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:35.813892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c57248-c6a0-4397-ab34-fdfc291f20d8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:35.841291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143aff3-cee9-4d16-b4f5-01cf0cfa7358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:35.842252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143aff3-cee9-4d16-b4f5-01cf0cfa7358 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.842743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143aff3-cee9-4d16-b4f5-01cf0cfa7358 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.843163 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143aff3-cee9-4d16-b4f5-01cf0cfa7358 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.844037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143aff3-cee9-4d16-b4f5-01cf0cfa7358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:35.844515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143aff3-cee9-4d16-b4f5-01cf0cfa7358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:35.846314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143aff3-cee9-4d16-b4f5-01cf0cfa7358 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:35.859386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 289/562] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:35.860429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143aff3-cee9-4d16-b4f5-01cf0cfa7358 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:35.925485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c57248-c6a0-4397-ab34-fdfc291f20d8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:35.953463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:35.965198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea95a8-b8fd-455e-87e1-56c7249838e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:35.966029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea95a8-b8fd-455e-87e1-56c7249838e8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:35.966297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea95a8-b8fd-455e-87e1-56c7249838e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:35.966576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea95a8-b8fd-455e-87e1-56c7249838e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:35.973663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143aff3-cee9-4d16-b4f5-01cf0cfa7358 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:35.980630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea95a8-b8fd-455e-87e1-56c7249838e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-07ea95a8-b8fd-455e-87e1-56c7249838e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:35.988674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:35.989745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 274/563] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:35.991300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea95a8-b8fd-455e-87e1-56c7249838e8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:35.997888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:36.017320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c2c4f-0569-44fb-acfe-d45a402df0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.017928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c2c4f-0569-44fb-acfe-d45a402df0ee None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.018214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c2c4f-0569-44fb-acfe-d45a402df0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.018495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c2c4f-0569-44fb-acfe-d45a402df0ee None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.019224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c2c4f-0569-44fb-acfe-d45a402df0ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:36.019561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c2c4f-0569-44fb-acfe-d45a402df0ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.021033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c2c4f-0569-44fb-acfe-d45a402df0ee None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.033451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 290/564] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:36.034294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c2c4f-0569-44fb-acfe-d45a402df0ee None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.103348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea95a8-b8fd-455e-87e1-56c7249838e8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.141944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c2c4f-0569-44fb-acfe-d45a402df0ee None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.144913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a2d26-101d-4a3b-9c2f-5e202a0f9176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.144913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a2d26-101d-4a3b-9c2f-5e202a0f9176 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.145212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a2d26-101d-4a3b-9c2f-5e202a0f9176 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.145452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a2d26-101d-4a3b-9c2f-5e202a0f9176 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.146310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a2d26-101d-4a3b-9c2f-5e202a0f9176 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:36.146799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a2d26-101d-4a3b-9c2f-5e202a0f9176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.148367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a2d26-101d-4a3b-9c2f-5e202a0f9176 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.161373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 275/565] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:36.161373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a2d26-101d-4a3b-9c2f-5e202a0f9176 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.185040 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0aff6-4d61-4c7e-9e57-4b28164bb037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.185844 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0aff6-4d61-4c7e-9e57-4b28164bb037 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.186126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0aff6-4d61-4c7e-9e57-4b28164bb037 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.186401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0aff6-4d61-4c7e-9e57-4b28164bb037 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.194305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0aff6-4d61-4c7e-9e57-4b28164bb037 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.195738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0aff6-4d61-4c7e-9e57-4b28164bb037 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.207866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-fcf0aff6-4d61-4c7e-9e57-4b28164bb037 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:36.210409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:36.211356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:36.212289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 291/566] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:36.212903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0aff6-4d61-4c7e-9e57-4b28164bb037 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.253671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:36.261369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a2d26-101d-4a3b-9c2f-5e202a0f9176 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.294634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b70c8-2c3d-4523-b2b9-995541a39e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.295234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b70c8-2c3d-4523-b2b9-995541a39e68 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.295517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b70c8-2c3d-4523-b2b9-995541a39e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.295781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b70c8-2c3d-4523-b2b9-995541a39e68 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.296570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b70c8-2c3d-4523-b2b9-995541a39e68 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:36.296873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b70c8-2c3d-4523-b2b9-995541a39e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.298242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b70c8-2c3d-4523-b2b9-995541a39e68 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.306127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 276/567] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:36.306911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b70c8-2c3d-4523-b2b9-995541a39e68 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.314190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0aff6-4d61-4c7e-9e57-4b28164bb037 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.347206 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27db9c-bf83-47e0-8004-e403ec117ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.348017 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27db9c-bf83-47e0-8004-e403ec117ebf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.348330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27db9c-bf83-47e0-8004-e403ec117ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.348608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27db9c-bf83-47e0-8004-e403ec117ebf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.355294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27db9c-bf83-47e0-8004-e403ec117ebf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.356785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27db9c-bf83-47e0-8004-e403ec117ebf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1c27db9c-bf83-47e0-8004-e403ec117ebf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:36.370351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:36.371347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:36.372267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:36.372267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:36.372267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:36.372267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:36.372267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:36.372267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:36.372267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:36.372267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:36.372267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 292/568] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:36.372741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b751cad-f769-4a19-b6c4-e513bdbfef30 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.373013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b751cad-f769-4a19-b6c4-e513bdbfef30 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.373263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b751cad-f769-4a19-b6c4-e513bdbfef30 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.374489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 293/569] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:36.378177 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b751cad-f769-4a19-b6c4-e513bdbfef30 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.412041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b70c8-2c3d-4523-b2b9-995541a39e68 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.424621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:36.445672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf72685-9160-4ecd-b013-7dd654412ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.446577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf72685-9160-4ecd-b013-7dd654412ddf None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.446857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf72685-9160-4ecd-b013-7dd654412ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.447127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf72685-9160-4ecd-b013-7dd654412ddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.457765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf72685-9160-4ecd-b013-7dd654412ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.459336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf72685-9160-4ecd-b013-7dd654412ddf None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.471412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-bbf72685-9160-4ecd-b013-7dd654412ddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:36.475991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:36.476925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:36.477819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:36.477819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:36.477819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:36.477819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:36.478047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b751cad-f769-4a19-b6c4-e513bdbfef30 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.478118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 277/570] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:36.478118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf72685-9160-4ecd-b013-7dd654412ddf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.513596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46322df-956f-45b1-b491-08043e32845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.514269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46322df-956f-45b1-b491-08043e32845c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.514549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46322df-956f-45b1-b491-08043e32845c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.514828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46322df-956f-45b1-b491-08043e32845c None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.515662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46322df-956f-45b1-b491-08043e32845c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:36.515964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46322df-956f-45b1-b491-08043e32845c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.517428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46322df-956f-45b1-b491-08043e32845c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.535973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 294/571] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:36.537018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46322df-956f-45b1-b491-08043e32845c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.587498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf72685-9160-4ecd-b013-7dd654412ddf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.623403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c986b-f101-4e45-99df-b632151b30dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.624238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c986b-f101-4e45-99df-b632151b30dd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.624520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c986b-f101-4e45-99df-b632151b30dd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.624792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c986b-f101-4e45-99df-b632151b30dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.636041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c986b-f101-4e45-99df-b632151b30dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.637670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c986b-f101-4e45-99df-b632151b30dd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.639747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46322df-956f-45b1-b491-08043e32845c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-0a2c986b-f101-4e45-99df-b632151b30dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:36.654335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:36.655327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:36.656777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:36.656777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:36.656777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:36.656777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:36.656777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 278/572] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:36.656777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbada85-b923-4e53-8502-071cae766c90 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.657183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbada85-b923-4e53-8502-071cae766c90 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.657262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbada85-b923-4e53-8502-071cae766c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.673182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3bbada85-b923-4e53-8502-071cae766c90 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:36.674370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3bbada85-b923-4e53-8502-071cae766c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:36.677184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd796db-fdff-48a7-a991-63218ada8bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.677925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd796db-fdff-48a7-a991-63218ada8bee None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.678207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd796db-fdff-48a7-a991-63218ada8bee None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.678480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd796db-fdff-48a7-a991-63218ada8bee None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.679266 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd796db-fdff-48a7-a991-63218ada8bee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:36.679572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd796db-fdff-48a7-a991-63218ada8bee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.681029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd796db-fdff-48a7-a991-63218ada8bee None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.685328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:36.692457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 295/573] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:36.693298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd796db-fdff-48a7-a991-63218ada8bee None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.711883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:36.744495 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 279/574] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:36.745330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbada85-b923-4e53-8502-071cae766c90 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.792472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd796db-fdff-48a7-a991-63218ada8bee None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.827598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24313a1f-bc41-4bcd-b3ab-d6e5b4144dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.828297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313a1f-bc41-4bcd-b3ab-d6e5b4144dae None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.828578 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313a1f-bc41-4bcd-b3ab-d6e5b4144dae None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.828848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313a1f-bc41-4bcd-b3ab-d6e5b4144dae None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.829557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24313a1f-bc41-4bcd-b3ab-d6e5b4144dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:36.829860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24313a1f-bc41-4bcd-b3ab-d6e5b4144dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.831248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24313a1f-bc41-4bcd-b3ab-d6e5b4144dae None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.842159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbada85-b923-4e53-8502-071cae766c90 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.843794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 296/575] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:36.844679 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24313a1f-bc41-4bcd-b3ab-d6e5b4144dae None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.881819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb92897-243f-436b-82e0-cfee47605cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.882628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb92897-243f-436b-82e0-cfee47605cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.882912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb92897-243f-436b-82e0-cfee47605cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.883188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb92897-243f-436b-82e0-cfee47605cf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.894010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb92897-243f-436b-82e0-cfee47605cf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.895495 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb92897-243f-436b-82e0-cfee47605cf8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:36.902887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-ddb92897-243f-436b-82e0-cfee47605cf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:36.916004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:36.917062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:36.918192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:36.918192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:36.918192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:36.918192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:36.918192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 280/576] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:36.918192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb92897-243f-436b-82e0-cfee47605cf8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:36.933935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:36.957409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24313a1f-bc41-4bcd-b3ab-d6e5b4144dae None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:36.992748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ec263-14fd-41dd-bb12-eff4b30dc62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:36.993421 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ec263-14fd-41dd-bb12-eff4b30dc62f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:36.993702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ec263-14fd-41dd-bb12-eff4b30dc62f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:36.993984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ec263-14fd-41dd-bb12-eff4b30dc62f None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:36.994714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ec263-14fd-41dd-bb12-eff4b30dc62f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:36.995023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ec263-14fd-41dd-bb12-eff4b30dc62f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:36.996435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ec263-14fd-41dd-bb12-eff4b30dc62f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:37.008028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 297/577] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:37.008888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ec263-14fd-41dd-bb12-eff4b30dc62f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.020540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb92897-243f-436b-82e0-cfee47605cf8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.055828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adc59c-0557-44ce-9cb2-d3c33e978034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.056534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adc59c-0557-44ce-9cb2-d3c33e978034 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.056814 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adc59c-0557-44ce-9cb2-d3c33e978034 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.057089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adc59c-0557-44ce-9cb2-d3c33e978034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.057931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6adc59c-0557-44ce-9cb2-d3c33e978034 None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=97b5f28d84fe4570a7c5c904dc4e502c) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:37.058243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6adc59c-0557-44ce-9cb2-d3c33e978034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:37.059680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6adc59c-0557-44ce-9cb2-d3c33e978034 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:37.065324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 281/578] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:36 2022] GET /identity/v3/role_assignments?user.id=97b5f28d84fe4570a7c5c904dc4e502c&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:37.066095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adc59c-0557-44ce-9cb2-d3c33e978034 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.116253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ec263-14fd-41dd-bb12-eff4b30dc62f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.145799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:37.155717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07121e-4e4b-44b6-9080-b680edf116b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.156895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07121e-4e4b-44b6-9080-b680edf116b8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.157380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07121e-4e4b-44b6-9080-b680edf116b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.157834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07121e-4e4b-44b6-9080-b680edf116b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.160997 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:37.170449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07121e-4e4b-44b6-9080-b680edf116b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-fe07121e-4e4b-44b6-9080-b680edf116b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:37.178756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:37.179774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 298/579] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:37.181471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07121e-4e4b-44b6-9080-b680edf116b8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.196559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adc59c-0557-44ce-9cb2-d3c33e978034 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.243821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03ff46-e612-47e8-b3e9-6ebf25f468d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.244685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03ff46-e612-47e8-b3e9-6ebf25f468d6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.244960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03ff46-e612-47e8-b3e9-6ebf25f468d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.245228 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03ff46-e612-47e8-b3e9-6ebf25f468d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.257291 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03ff46-e612-47e8-b3e9-6ebf25f468d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:37.258765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03ff46-e612-47e8-b3e9-6ebf25f468d6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-ac03ff46-e612-47e8-b3e9-6ebf25f468d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:37.273527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:37.274543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 282/580] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:37.280451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03ff46-e612-47e8-b3e9-6ebf25f468d6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.305862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07121e-4e4b-44b6-9080-b680edf116b8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.340417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363cee1-edc7-4aa7-966f-2e58c116a6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.341254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363cee1-edc7-4aa7-966f-2e58c116a6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.341624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363cee1-edc7-4aa7-966f-2e58c116a6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.341985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363cee1-edc7-4aa7-966f-2e58c116a6c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.342768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363cee1-edc7-4aa7-966f-2e58c116a6c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:37.343155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363cee1-edc7-4aa7-966f-2e58c116a6c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:37.344681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363cee1-edc7-4aa7-966f-2e58c116a6c4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:37.357604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 299/581] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:37.358441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363cee1-edc7-4aa7-966f-2e58c116a6c4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.389323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03ff46-e612-47e8-b3e9-6ebf25f468d6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.415608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:37.422855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:37.431422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413516b1-17e3-40fd-8bd3-192a6bd7a537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.432287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413516b1-17e3-40fd-8bd3-192a6bd7a537 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.432572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413516b1-17e3-40fd-8bd3-192a6bd7a537 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.432842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413516b1-17e3-40fd-8bd3-192a6bd7a537 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.440201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413516b1-17e3-40fd-8bd3-192a6bd7a537 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:37.441652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413516b1-17e3-40fd-8bd3-192a6bd7a537 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-413516b1-17e3-40fd-8bd3-192a6bd7a537 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:37.457645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:37.458666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 283/582] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:37.461184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413516b1-17e3-40fd-8bd3-192a6bd7a537 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.473317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363cee1-edc7-4aa7-966f-2e58c116a6c4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.509008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97565be7-b1c0-4b95-9863-600ff1e5dfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.509985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565be7-b1c0-4b95-9863-600ff1e5dfbb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.510265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565be7-b1c0-4b95-9863-600ff1e5dfbb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.510536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565be7-b1c0-4b95-9863-600ff1e5dfbb None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.511312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97565be7-b1c0-4b95-9863-600ff1e5dfbb None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:37.511602 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97565be7-b1c0-4b95-9863-600ff1e5dfbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:37.513035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97565be7-b1c0-4b95-9863-600ff1e5dfbb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:37.526049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 300/583] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:37.526049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97565be7-b1c0-4b95-9863-600ff1e5dfbb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.585981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413516b1-17e3-40fd-8bd3-192a6bd7a537 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.622775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbd6a6-9a30-4236-9b07-3055072ce5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.623692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbd6a6-9a30-4236-9b07-3055072ce5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.624301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbd6a6-9a30-4236-9b07-3055072ce5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.624580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbd6a6-9a30-4236-9b07-3055072ce5f7 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.625691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbd6a6-9a30-4236-9b07-3055072ce5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:37.626019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbd6a6-9a30-4236-9b07-3055072ce5f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:37.628157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbd6a6-9a30-4236-9b07-3055072ce5f7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:37.633619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97565be7-b1c0-4b95-9863-600ff1e5dfbb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.651476 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:37.652045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 284/584] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:37.652888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbd6a6-9a30-4236-9b07-3055072ce5f7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.669895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfcf36-c72b-4c92-a23d-8625fb0f5bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.670705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfcf36-c72b-4c92-a23d-8625fb0f5bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.670983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfcf36-c72b-4c92-a23d-8625fb0f5bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.671255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfcf36-c72b-4c92-a23d-8625fb0f5bf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.678425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfcf36-c72b-4c92-a23d-8625fb0f5bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:37.679979 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfcf36-c72b-4c92-a23d-8625fb0f5bf4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-34bfcf36-c72b-4c92-a23d-8625fb0f5bf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:37.694707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:37.695843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:37.697007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:37.697007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:37.697007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:37.697007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:37.697007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:37.697007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:37.697007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:37.697007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:37.698089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 301/585] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:37.698946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfcf36-c72b-4c92-a23d-8625fb0f5bf4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.705428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:37.772139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbd6a6-9a30-4236-9b07-3055072ce5f7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.808406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfcf36-c72b-4c92-a23d-8625fb0f5bf4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.810360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e4690-ac7b-4b4f-973d-111f9b8ac7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.811799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e4690-ac7b-4b4f-973d-111f9b8ac7ae None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.812082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e4690-ac7b-4b4f-973d-111f9b8ac7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.812380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e4690-ac7b-4b4f-973d-111f9b8ac7ae None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.813227 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e4690-ac7b-4b4f-973d-111f9b8ac7ae None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:37.813532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e4690-ac7b-4b4f-973d-111f9b8ac7ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:37.817259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e4690-ac7b-4b4f-973d-111f9b8ac7ae None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:37.839723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 285/586] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:37.839723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e4690-ac7b-4b4f-973d-111f9b8ac7ae None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.852268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add7a12-afa4-445d-a536-84ea8fc2cabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.853090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7a12-afa4-445d-a536-84ea8fc2cabd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.853445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7a12-afa4-445d-a536-84ea8fc2cabd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.853814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7a12-afa4-445d-a536-84ea8fc2cabd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.864692 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add7a12-afa4-445d-a536-84ea8fc2cabd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:37.866447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add7a12-afa4-445d-a536-84ea8fc2cabd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1add7a12-afa4-445d-a536-84ea8fc2cabd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:37.884576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:37.885613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:37.886686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:37.886686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:37.886686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:37.886686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:37.890702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 302/587] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:37.891541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add7a12-afa4-445d-a536-84ea8fc2cabd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:37.924016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:37.944775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:37.949471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e4690-ac7b-4b4f-973d-111f9b8ac7ae None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.985516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0cba6-9b55-4d12-bbee-4bbabf6c7e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:37.986565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0cba6-9b55-4d12-bbee-4bbabf6c7e51 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:37.987019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0cba6-9b55-4d12-bbee-4bbabf6c7e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:37.987348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0cba6-9b55-4d12-bbee-4bbabf6c7e51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:37.996240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add7a12-afa4-445d-a536-84ea8fc2cabd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:37.997782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0cba6-9b55-4d12-bbee-4bbabf6c7e51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:37.999237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0cba6-9b55-4d12-bbee-4bbabf6c7e51 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-6fc0cba6-9b55-4d12-bbee-4bbabf6c7e51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:38.017444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:38.018370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:38.019804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:38.019804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:38.019804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:38.019804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:38.019804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 286/588] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:38.019804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0cba6-9b55-4d12-bbee-4bbabf6c7e51 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:38.037673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7afdc4-b14e-4830-9627-c925f902758c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.038338 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7afdc4-b14e-4830-9627-c925f902758c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.038618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7afdc4-b14e-4830-9627-c925f902758c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.038886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7afdc4-b14e-4830-9627-c925f902758c None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.039675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7afdc4-b14e-4830-9627-c925f902758c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:38.039978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7afdc4-b14e-4830-9627-c925f902758c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.041486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7afdc4-b14e-4830-9627-c925f902758c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.053909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 303/589] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:38.059283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7afdc4-b14e-4830-9627-c925f902758c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:38.124214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0cba6-9b55-4d12-bbee-4bbabf6c7e51 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:38.159437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:38.162435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af1be7-b486-4660-bf90-799c82d03b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.163356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af1be7-b486-4660-bf90-799c82d03b65 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.163644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af1be7-b486-4660-bf90-799c82d03b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.163917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af1be7-b486-4660-bf90-799c82d03b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.172155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7afdc4-b14e-4830-9627-c925f902758c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:38.175917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af1be7-b486-4660-bf90-799c82d03b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.177453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af1be7-b486-4660-bf90-799c82d03b65 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d5af1be7-b486-4660-bf90-799c82d03b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:38.194965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:38.195879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:38.196771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:38.196771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:38.196771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:38.196771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:38.200405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 287/590] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:38.201783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af1be7-b486-4660-bf90-799c82d03b65 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:38.204984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:38.213961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46d078-1dc5-4a34-9371-ab30d40785c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.214955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46d078-1dc5-4a34-9371-ab30d40785c9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.215246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46d078-1dc5-4a34-9371-ab30d40785c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.215678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46d078-1dc5-4a34-9371-ab30d40785c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.216443 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46d078-1dc5-4a34-9371-ab30d40785c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:38.216755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46d078-1dc5-4a34-9371-ab30d40785c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.218158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46d078-1dc5-4a34-9371-ab30d40785c9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.230383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 304/591] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:38.231164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46d078-1dc5-4a34-9371-ab30d40785c9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:38.312804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af1be7-b486-4660-bf90-799c82d03b65 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:38.343818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46d078-1dc5-4a34-9371-ab30d40785c9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:38.348412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfb6c2-be5f-49af-894a-043468a8436b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.349084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfb6c2-be5f-49af-894a-043468a8436b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.349419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfb6c2-be5f-49af-894a-043468a8436b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.349698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfb6c2-be5f-49af-894a-043468a8436b None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.350430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfb6c2-be5f-49af-894a-043468a8436b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:38.350738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfb6c2-be5f-49af-894a-043468a8436b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.352182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfb6c2-be5f-49af-894a-043468a8436b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.368387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 288/592] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:38.368387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfb6c2-be5f-49af-894a-043468a8436b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:38.378195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d3d1c-8c83-45e3-b749-bf523864d358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.378773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d3d1c-8c83-45e3-b749-bf523864d358 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.379032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d3d1c-8c83-45e3-b749-bf523864d358 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.379301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d3d1c-8c83-45e3-b749-bf523864d358 None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/097b12fbf6494336b11ece5c498a4103/roles/91ff1620e1064bf199221383f124aa3c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.395312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d3d1c-8c83-45e3-b749-bf523864d358 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=097b12fbf6494336b11ece5c498a4103, role_id=91ff1620e1064bf199221383f124aa3c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.396820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d3d1c-8c83-45e3-b749-bf523864d358 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.408682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:38.413477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 305/593] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:44:38 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/097b12fbf6494336b11ece5c498a4103/roles/91ff1620e1064bf199221383f124aa3c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:38.425182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d3d1c-8c83-45e3-b749-bf523864d358 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:38.441185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:38.500542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfb6c2-be5f-49af-894a-043468a8436b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:38.553109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c326fc0-4c38-45d0-9ec0-218c74caec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.553412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d3d1c-8c83-45e3-b749-bf523864d358 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:38.554312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c326fc0-4c38-45d0-9ec0-218c74caec58 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.554775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c326fc0-4c38-45d0-9ec0-218c74caec58 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.555528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c326fc0-4c38-45d0-9ec0-218c74caec58 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.557537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c326fc0-4c38-45d0-9ec0-218c74caec58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:38.558062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c326fc0-4c38-45d0-9ec0-218c74caec58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.559696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c326fc0-4c38-45d0-9ec0-218c74caec58 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.578011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 289/594] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:38.579181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c326fc0-4c38-45d0-9ec0-218c74caec58 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:38.609559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deea08a-8c5a-451f-9929-45a9d5d9e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.610789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deea08a-8c5a-451f-9929-45a9d5d9e0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.611270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deea08a-8c5a-451f-9929-45a9d5d9e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.611739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deea08a-8c5a-451f-9929-45a9d5d9e0f0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.620416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deea08a-8c5a-451f-9929-45a9d5d9e0f0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.622157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deea08a-8c5a-451f-9929-45a9d5d9e0f0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9deea08a-8c5a-451f-9929-45a9d5d9e0f0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:38.637641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:38.638619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:38.639543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:38.639543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:38.639543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:38.639543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:38.639543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:38.639543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:38.639543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:38.639543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:38.639543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 20:44:38.639543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:38.641895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 306/595] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:38.643037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deea08a-8c5a-451f-9929-45a9d5d9e0f0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:38.706940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:38.708569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c326fc0-4c38-45d0-9ec0-218c74caec58 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:38.750579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:38.755800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deea08a-8c5a-451f-9929-45a9d5d9e0f0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:38.761199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69277475-c6d9-47cf-89f2-926dd06b3206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.762352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277475-c6d9-47cf-89f2-926dd06b3206 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.762835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277475-c6d9-47cf-89f2-926dd06b3206 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.763333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277475-c6d9-47cf-89f2-926dd06b3206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.776045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69277475-c6d9-47cf-89f2-926dd06b3206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.777924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69277475-c6d9-47cf-89f2-926dd06b3206 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.793114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29eed6-f83c-4b38-b836-53fa91f2d021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.794112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29eed6-f83c-4b38-b836-53fa91f2d021 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.794394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29eed6-f83c-4b38-b836-53fa91f2d021 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.794666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29eed6-f83c-4b38-b836-53fa91f2d021 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.801662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29eed6-f83c-4b38-b836-53fa91f2d021 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-69277475-c6d9-47cf-89f2-926dd06b3206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:38.801901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:38.803076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:38.804159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:38.804159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:38.804159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:38.804159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:38.804159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:38.804159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:38.804159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:38.804159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:38.804608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29eed6-f83c-4b38-b836-53fa91f2d021 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.807268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 290/596] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:38.808342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69277475-c6d9-47cf-89f2-926dd06b3206 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-eb29eed6-f83c-4b38-b836-53fa91f2d021 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:38.817490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:38.818762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:38.819827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:38.819827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:38.819827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:38.819827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:38.819827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:38.819827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:38.819827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:38.819827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:38.819827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 307/597] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:38.820422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29eed6-f83c-4b38-b836-53fa91f2d021 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:38.939330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69277475-c6d9-47cf-89f2-926dd06b3206 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:38.941621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29eed6-f83c-4b38-b836-53fa91f2d021 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:38.968283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:38.981493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e197efe-5e71-4112-a346-a166c8de7d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.982589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e197efe-5e71-4112-a346-a166c8de7d18 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.982708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e197efe-5e71-4112-a346-a166c8de7d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.983067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e197efe-5e71-4112-a346-a166c8de7d18 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.984033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e197efe-5e71-4112-a346-a166c8de7d18 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:38.984538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e197efe-5e71-4112-a346-a166c8de7d18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.986390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e197efe-5e71-4112-a346-a166c8de7d18 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:38.992157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdf7ce-eda8-467c-b155-5971d3f3eb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:38.992157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdf7ce-eda8-467c-b155-5971d3f3eb18 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:38.992157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdf7ce-eda8-467c-b155-5971d3f3eb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:38.992157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdf7ce-eda8-467c-b155-5971d3f3eb18 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:38.993240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffdf7ce-eda8-467c-b155-5971d3f3eb18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:38.993761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffdf7ce-eda8-467c-b155-5971d3f3eb18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:38.995276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffdf7ce-eda8-467c-b155-5971d3f3eb18 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:39.015082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 308/598] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:39.015082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdf7ce-eda8-467c-b155-5971d3f3eb18 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:39.015776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 291/599] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Jul 11 20:44:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:39.017102 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e197efe-5e71-4112-a346-a166c8de7d18 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:39.065230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:39.126198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdf7ce-eda8-467c-b155-5971d3f3eb18 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:39.130524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e197efe-5e71-4112-a346-a166c8de7d18 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:39.163463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd65158-b0df-43b7-a9bc-95067be6581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:39.164349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd65158-b0df-43b7-a9bc-95067be6581b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:39.164482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd65158-b0df-43b7-a9bc-95067be6581b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:39.164707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd65158-b0df-43b7-a9bc-95067be6581b None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:39.165524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd65158-b0df-43b7-a9bc-95067be6581b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:39.165804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd65158-b0df-43b7-a9bc-95067be6581b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:39.167242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd65158-b0df-43b7-a9bc-95067be6581b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:39.169503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea78a5-03c9-4747-99dd-a6ac6b8a9425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:39.170340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea78a5-03c9-4747-99dd-a6ac6b8a9425 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:39.170583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea78a5-03c9-4747-99dd-a6ac6b8a9425 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:39.170863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea78a5-03c9-4747-99dd-a6ac6b8a9425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:39.184151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 309/600] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:39.184151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd65158-b0df-43b7-a9bc-95067be6581b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:39.187767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea78a5-03c9-4747-99dd-a6ac6b8a9425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:39.189293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea78a5-03c9-4747-99dd-a6ac6b8a9425 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-98ea78a5-03c9-4747-99dd-a6ac6b8a9425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:39.208998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:39.210100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:39.215679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:39.215679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:39.215679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:39.215679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:39.215679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 292/601] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:39.215679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f28bd-b03d-43ce-90a3-e5dffd89b78e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:39.215679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f28bd-b03d-43ce-90a3-e5dffd89b78e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:39.215679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f28bd-b03d-43ce-90a3-e5dffd89b78e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:39.215679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 293/602] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:39.216477 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f28bd-b03d-43ce-90a3-e5dffd89b78e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:39.236550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:39.297586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd65158-b0df-43b7-a9bc-95067be6581b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:39.329616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:39.342445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f28bd-b03d-43ce-90a3-e5dffd89b78e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:39.342601 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a10fc6-8d49-4a38-b2b6-710078d65a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:39.343490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a10fc6-8d49-4a38-b2b6-710078d65a9e None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:39.343914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a10fc6-8d49-4a38-b2b6-710078d65a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:39.344504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a10fc6-8d49-4a38-b2b6-710078d65a9e None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/acb3ad858e4848b9a325c72c7328c863/roles/91ff1620e1064bf199221383f124aa3c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:39.361122 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a10fc6-8d49-4a38-b2b6-710078d65a9e None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=acb3ad858e4848b9a325c72c7328c863, role_id=91ff1620e1064bf199221383f124aa3c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:39.362445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a10fc6-8d49-4a38-b2b6-710078d65a9e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:39.384147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 310/603] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:44:39 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/acb3ad858e4848b9a325c72c7328c863/roles/91ff1620e1064bf199221383f124aa3c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:39.384147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a10fc6-8d49-4a38-b2b6-710078d65a9e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:39.385437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b7059-a847-4a62-88b7-b00196136c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:39.386501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7059-a847-4a62-88b7-b00196136c6e None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:39.386878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7059-a847-4a62-88b7-b00196136c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:39.387216 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7059-a847-4a62-88b7-b00196136c6e None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/dee928cd256446cb9e000ceb6e9ef636/roles/91ff1620e1064bf199221383f124aa3c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:39.406952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b7059-a847-4a62-88b7-b00196136c6e None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=dee928cd256446cb9e000ceb6e9ef636, role_id=91ff1620e1064bf199221383f124aa3c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:39.408479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b7059-a847-4a62-88b7-b00196136c6e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:39.436235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 294/604] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:44:39 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/dee928cd256446cb9e000ceb6e9ef636/roles/91ff1620e1064bf199221383f124aa3c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:39.437972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4624f7-0a3a-4882-b2db-2afd9fed5d65 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:39.438350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4624f7-0a3a-4882-b2db-2afd9fed5d65 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:39.438696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4624f7-0a3a-4882-b2db-2afd9fed5d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:39.476751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9f4624f7-0a3a-4882-b2db-2afd9fed5d65 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:39.479059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9f4624f7-0a3a-4882-b2db-2afd9fed5d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:39.548994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:39.565888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a10fc6-8d49-4a38-b2b6-710078d65a9e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:39.600862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 295/605] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:39.611853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c20bc-9e08-4c8f-b58b-1492a3519dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:39.612640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c20bc-9e08-4c8f-b58b-1492a3519dac None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:39.612925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c20bc-9e08-4c8f-b58b-1492a3519dac None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:39.613201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c20bc-9e08-4c8f-b58b-1492a3519dac None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:39.613990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c20bc-9e08-4c8f-b58b-1492a3519dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:39.614350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c20bc-9e08-4c8f-b58b-1492a3519dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:39.616348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c20bc-9e08-4c8f-b58b-1492a3519dac None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:39.629790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4624f7-0a3a-4882-b2db-2afd9fed5d65 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:39.634134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 311/606] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:39.640350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c20bc-9e08-4c8f-b58b-1492a3519dac None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:39.724249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:39.766972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4624f7-0a3a-4882-b2db-2afd9fed5d65 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:39.780473 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c20bc-9e08-4c8f-b58b-1492a3519dac None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:39.839976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0dcaa-bab0-4457-a2e5-df4a12e7da76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:39.840827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0dcaa-bab0-4457-a2e5-df4a12e7da76 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:39.841106 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0dcaa-bab0-4457-a2e5-df4a12e7da76 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:39.841376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0dcaa-bab0-4457-a2e5-df4a12e7da76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:39.844090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9a39b-3184-4dcc-bf7d-4f943157ef44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:39.844922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9a39b-3184-4dcc-bf7d-4f943157ef44 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:39.845196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9a39b-3184-4dcc-bf7d-4f943157ef44 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:39.845466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9a39b-3184-4dcc-bf7d-4f943157ef44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:39.860736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0dcaa-bab0-4457-a2e5-df4a12e7da76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:39.868140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9a39b-3184-4dcc-bf7d-4f943157ef44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:39.869634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9a39b-3184-4dcc-bf7d-4f943157ef44 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-3ad0dcaa-bab0-4457-a2e5-df4a12e7da76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:39.871385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:39.872320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:39.873144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 296/607] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:39.880505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0dcaa-bab0-4457-a2e5-df4a12e7da76 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-eac9a39b-3184-4dcc-bf7d-4f943157ef44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:39.900451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:39.901371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:39.902279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:39.902279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:39.902279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:39.902279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:39.902279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:39.902279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:39.902279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:39.902279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:39.904913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 312/608] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:39.910523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9a39b-3184-4dcc-bf7d-4f943157ef44 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:39.928714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:40.048134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0dcaa-bab0-4457-a2e5-df4a12e7da76 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:40.084053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9a39b-3184-4dcc-bf7d-4f943157ef44 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:40.096471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:40.103670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0db773-d8a4-4975-9b06-c1e448bffb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:40.104391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0db773-d8a4-4975-9b06-c1e448bffb1f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:40.104676 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0db773-d8a4-4975-9b06-c1e448bffb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:40.104949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0db773-d8a4-4975-9b06-c1e448bffb1f None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:40.105721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0db773-d8a4-4975-9b06-c1e448bffb1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:40.106033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0db773-d8a4-4975-9b06-c1e448bffb1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:40.107427 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0db773-d8a4-4975-9b06-c1e448bffb1f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:40.120992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 297/609] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:40.126498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0db773-d8a4-4975-9b06-c1e448bffb1f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:40.152916 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b8ae8-adc3-43f8-a55b-358f50c28353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:40.153994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b8ae8-adc3-43f8-a55b-358f50c28353 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:40.154460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b8ae8-adc3-43f8-a55b-358f50c28353 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:40.154913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b8ae8-adc3-43f8-a55b-358f50c28353 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:40.155907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b8ae8-adc3-43f8-a55b-358f50c28353 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:40.156434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b8ae8-adc3-43f8-a55b-358f50c28353 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:40.158202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b8ae8-adc3-43f8-a55b-358f50c28353 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:40.178681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 313/610] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:40.192159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b8ae8-adc3-43f8-a55b-358f50c28353 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:40.264797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0db773-d8a4-4975-9b06-c1e448bffb1f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:40.296128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d791776-a2c9-4e95-8bf9-3380780a4251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:40.296128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d791776-a2c9-4e95-8bf9-3380780a4251 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:40.296128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d791776-a2c9-4e95-8bf9-3380780a4251 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:40.296498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d791776-a2c9-4e95-8bf9-3380780a4251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:40.316245 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d791776-a2c9-4e95-8bf9-3380780a4251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:40.316245 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d791776-a2c9-4e95-8bf9-3380780a4251 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:40.330960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-2d791776-a2c9-4e95-8bf9-3380780a4251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:40.331173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:40.332165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:40.333233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:40.333233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:40.333233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:40.333233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:40.333233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 298/611] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:40.335999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d791776-a2c9-4e95-8bf9-3380780a4251 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:40.383286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b8ae8-adc3-43f8-a55b-358f50c28353 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:40.388586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:40.442708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d791776-a2c9-4e95-8bf9-3380780a4251 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:40.451273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24242b-4405-4612-9bac-81f8e775910a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:40.452590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24242b-4405-4612-9bac-81f8e775910a None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:40.453107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24242b-4405-4612-9bac-81f8e775910a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:40.453508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24242b-4405-4612-9bac-81f8e775910a None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/0574d5c8974f43218653208ca72c43bb/roles/91ff1620e1064bf199221383f124aa3c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:40.477856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24242b-4405-4612-9bac-81f8e775910a None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=0574d5c8974f43218653208ca72c43bb, role_id=91ff1620e1064bf199221383f124aa3c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:40.479320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24242b-4405-4612-9bac-81f8e775910a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:40.483845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea044-a517-4778-a50b-5969823f6e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:40.484556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea044-a517-4778-a50b-5969823f6e53 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:40.484835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea044-a517-4778-a50b-5969823f6e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:40.485168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea044-a517-4778-a50b-5969823f6e53 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:40.485910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fea044-a517-4778-a50b-5969823f6e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:40.487106 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fea044-a517-4778-a50b-5969823f6e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:40.488576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fea044-a517-4778-a50b-5969823f6e53 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:40.508374 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 299/613] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:40.512287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 314/613] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:44:40 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/0574d5c8974f43218653208ca72c43bb/roles/91ff1620e1064bf199221383f124aa3c => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:40.513512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea044-a517-4778-a50b-5969823f6e53 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:40.608246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2813bb9-74e2-44a1-aa88-139feac7073f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:40.608598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2813bb9-74e2-44a1-aa88-139feac7073f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:40.608812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2813bb9-74e2-44a1-aa88-139feac7073f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:40.610359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 315/614] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:40.616079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adddac3-d84a-47d0-8936-a88b7e0c6ec3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:40.616429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adddac3-d84a-47d0-8936-a88b7e0c6ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:40.616658 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adddac3-d84a-47d0-8936-a88b7e0c6ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:40.653900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4adddac3-d84a-47d0-8936-a88b7e0c6ec3 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:40.655595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4adddac3-d84a-47d0-8936-a88b7e0c6ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:40.699776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea044-a517-4778-a50b-5969823f6e53 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:40.709889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:40.742100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c3eea-7378-4b35-9c54-8ce9a461a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:40.743504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c3eea-7378-4b35-9c54-8ce9a461a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:40.744141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c3eea-7378-4b35-9c54-8ce9a461a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:40.744937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c3eea-7378-4b35-9c54-8ce9a461a3b8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:40.755333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 316/615] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:40.758707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c3eea-7378-4b35-9c54-8ce9a461a3b8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:40.760324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c3eea-7378-4b35-9c54-8ce9a461a3b8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c91c3eea-7378-4b35-9c54-8ce9a461a3b8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:40.780666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:40.781713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:40.782615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 300/616] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:40.788043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adddac3-d84a-47d0-8936-a88b7e0c6ec3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:40.798206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c3eea-7378-4b35-9c54-8ce9a461a3b8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:40.920583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:40.940750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c3eea-7378-4b35-9c54-8ce9a461a3b8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:40.993328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adddac3-d84a-47d0-8936-a88b7e0c6ec3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:41.009991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a1f5e-4eb7-4045-aee1-8150567a94c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:41.010809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a1f5e-4eb7-4045-aee1-8150567a94c1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:41.011086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a1f5e-4eb7-4045-aee1-8150567a94c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:41.011353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a1f5e-4eb7-4045-aee1-8150567a94c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:41.030211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a1f5e-4eb7-4045-aee1-8150567a94c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f24a1f5e-4eb7-4045-aee1-8150567a94c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:41.041928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:41.042834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:41.043672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:41.047916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 301/617] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:41.052291 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a1f5e-4eb7-4045-aee1-8150567a94c1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:41.055447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04b04a-a7ab-4e4c-ac34-72d06a9c26ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:41.057302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04b04a-a7ab-4e4c-ac34-72d06a9c26ef None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:41.057584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04b04a-a7ab-4e4c-ac34-72d06a9c26ef None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:41.057847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04b04a-a7ab-4e4c-ac34-72d06a9c26ef None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:41.059587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04b04a-a7ab-4e4c-ac34-72d06a9c26ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:41.059896 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04b04a-a7ab-4e4c-ac34-72d06a9c26ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:41.061370 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04b04a-a7ab-4e4c-ac34-72d06a9c26ef None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:41.088328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 317/618] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jul 11 20:44:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:41.093223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04b04a-a7ab-4e4c-ac34-72d06a9c26ef None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:41.113341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:41.254163 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04b04a-a7ab-4e4c-ac34-72d06a9c26ef None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:41.264015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:41.265184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a1f5e-4eb7-4045-aee1-8150567a94c1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:41.309023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782bd9f-a330-4185-8bdc-4a133e62176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:41.310206 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782bd9f-a330-4185-8bdc-4a133e62176c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:41.310675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782bd9f-a330-4185-8bdc-4a133e62176c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:41.311256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782bd9f-a330-4185-8bdc-4a133e62176c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:41.328540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782bd9f-a330-4185-8bdc-4a133e62176c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:41.330472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782bd9f-a330-4185-8bdc-4a133e62176c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:41.337036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fd5ae-cb1f-4ba0-a043-149be5b2e2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:41.337976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fd5ae-cb1f-4ba0-a043-149be5b2e2dc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:41.338475 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fd5ae-cb1f-4ba0-a043-149be5b2e2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:41.338988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fd5ae-cb1f-4ba0-a043-149be5b2e2dc None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:41.339934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7fd5ae-cb1f-4ba0-a043-149be5b2e2dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:41.340472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7fd5ae-cb1f-4ba0-a043-149be5b2e2dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:41.342020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7fd5ae-cb1f-4ba0-a043-149be5b2e2dc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:41.366739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 302/619] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c782bd9f-a330-4185-8bdc-4a133e62176c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:41.367716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:41.368747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:41.369624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:41.369624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:41.369624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:41.369624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:41.369624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 318/620] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:41.374436 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fd5ae-cb1f-4ba0-a043-149be5b2e2dc None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:41.376042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782bd9f-a330-4185-8bdc-4a133e62176c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:41.487522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782bd9f-a330-4185-8bdc-4a133e62176c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:41.533367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb101c7-0d04-4c3e-82b2-155bf917fc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:41.534001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb101c7-0d04-4c3e-82b2-155bf917fc2c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:41.534353 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb101c7-0d04-4c3e-82b2-155bf917fc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:41.534628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb101c7-0d04-4c3e-82b2-155bf917fc2c None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:41.535371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb101c7-0d04-4c3e-82b2-155bf917fc2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:41.535738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb101c7-0d04-4c3e-82b2-155bf917fc2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:41.537475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb101c7-0d04-4c3e-82b2-155bf917fc2c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:41.553534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:41.557767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fd5ae-cb1f-4ba0-a043-149be5b2e2dc None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:41.564222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 319/621] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:41.566148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb101c7-0d04-4c3e-82b2-155bf917fc2c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:41.593474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e971ca2-c087-435b-9800-562d42ebecd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:41.594384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971ca2-c087-435b-9800-562d42ebecd1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:41.594740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971ca2-c087-435b-9800-562d42ebecd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:41.595107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971ca2-c087-435b-9800-562d42ebecd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:41.603889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:41.609660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e971ca2-c087-435b-9800-562d42ebecd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:41.616424 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e971ca2-c087-435b-9800-562d42ebecd1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-5e971ca2-c087-435b-9800-562d42ebecd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:41.630998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:41.631890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:41.633129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:41.633129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:41.633129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:41.633129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:41.633129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 303/622] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:41.636584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e971ca2-c087-435b-9800-562d42ebecd1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:41.746443 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e971ca2-c087-435b-9800-562d42ebecd1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:41.785692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af71c9f-def7-4236-b9c4-e01f3b93bb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:41.786427 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af71c9f-def7-4236-b9c4-e01f3b93bb75 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:41.786766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af71c9f-def7-4236-b9c4-e01f3b93bb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:41.787048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af71c9f-def7-4236-b9c4-e01f3b93bb75 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:41.787795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af71c9f-def7-4236-b9c4-e01f3b93bb75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:41.788144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af71c9f-def7-4236-b9c4-e01f3b93bb75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:41.789797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af71c9f-def7-4236-b9c4-e01f3b93bb75 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:41.803442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 304/623] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:41.804375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:41.808283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb101c7-0d04-4c3e-82b2-155bf917fc2c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:41.809056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af71c9f-def7-4236-b9c4-e01f3b93bb75 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:41.869381 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770fc40-67b6-4e17-8ee1-8e1ee45730f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:41.870569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770fc40-67b6-4e17-8ee1-8e1ee45730f6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:41.871071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770fc40-67b6-4e17-8ee1-8e1ee45730f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:41.871604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770fc40-67b6-4e17-8ee1-8e1ee45730f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:41.883416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770fc40-67b6-4e17-8ee1-8e1ee45730f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:41.885222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770fc40-67b6-4e17-8ee1-8e1ee45730f6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7770fc40-67b6-4e17-8ee1-8e1ee45730f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:41.900847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:41.901714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:41.902569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:41.902569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:41.902569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:41.902569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:41.902569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:41.902569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:41.902569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:41.902569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:41.905178 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 320/624] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:41.906624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a772146-e8be-4d68-903a-c57e5440e51a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:41.907165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a772146-e8be-4d68-903a-c57e5440e51a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:41.907630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a772146-e8be-4d68-903a-c57e5440e51a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:41.909445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 321/625] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:41.913069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a772146-e8be-4d68-903a-c57e5440e51a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:41.935630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af71c9f-def7-4236-b9c4-e01f3b93bb75 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:41.971140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57d40a-1fd3-488b-9d51-c798a3e62d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:41.971948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57d40a-1fd3-488b-9d51-c798a3e62d18 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:41.972545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57d40a-1fd3-488b-9d51-c798a3e62d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:41.972674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57d40a-1fd3-488b-9d51-c798a3e62d18 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:41.980395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57d40a-1fd3-488b-9d51-c798a3e62d18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:41.981908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57d40a-1fd3-488b-9d51-c798a3e62d18 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-1b57d40a-1fd3-488b-9d51-c798a3e62d18 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:41.997075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:41.998044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:41.999582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 305/626] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:42.000188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd29a5-c0e4-4e82-90ca-ba8720cdde6e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.000188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd29a5-c0e4-4e82-90ca-ba8720cdde6e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.000533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd29a5-c0e4-4e82-90ca-ba8720cdde6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.006314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:42.017248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e6bd29a5-c0e4-4e82-90ca-ba8720cdde6e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:42.018498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e6bd29a5-c0e4-4e82-90ca-ba8720cdde6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:42.029929 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a772146-e8be-4d68-903a-c57e5440e51a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:42.067410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ecd31-ae9c-419a-a0c3-59e59b9d5391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:42.069035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ecd31-ae9c-419a-a0c3-59e59b9d5391 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.069525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ecd31-ae9c-419a-a0c3-59e59b9d5391 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.069995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ecd31-ae9c-419a-a0c3-59e59b9d5391 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.071111 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ecd31-ae9c-419a-a0c3-59e59b9d5391 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:42.071655 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ecd31-ae9c-419a-a0c3-59e59b9d5391 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:42.073382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ecd31-ae9c-419a-a0c3-59e59b9d5391 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:42.078284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:42.088391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 322/627] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:42.089569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ecd31-ae9c-419a-a0c3-59e59b9d5391 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:42.094555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 306/628] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:42.095589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd29a5-c0e4-4e82-90ca-ba8720cdde6e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:42.195180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ecd31-ae9c-419a-a0c3-59e59b9d5391 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:42.196289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd29a5-c0e4-4e82-90ca-ba8720cdde6e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:42.231071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d788f39-4300-4281-b99a-0951628cec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:42.231749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d788f39-4300-4281-b99a-0951628cec25 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.231955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f860bca-a501-440e-9b30-2ece477419a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:42.232070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d788f39-4300-4281-b99a-0951628cec25 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.232324 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d788f39-4300-4281-b99a-0951628cec25 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.232880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f860bca-a501-440e-9b30-2ece477419a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.233156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f860bca-a501-440e-9b30-2ece477419a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.233247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d788f39-4300-4281-b99a-0951628cec25 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:42.233424 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f860bca-a501-440e-9b30-2ece477419a4 None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/97b5f28d84fe4570a7c5c904dc4e502c/roles/91ff1620e1064bf199221383f124aa3c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.233527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d788f39-4300-4281-b99a-0951628cec25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:42.234899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d788f39-4300-4281-b99a-0951628cec25 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:42.245382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:42.249089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f860bca-a501-440e-9b30-2ece477419a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=97b5f28d84fe4570a7c5c904dc4e502c, role_id=91ff1620e1064bf199221383f124aa3c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:42.250538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f860bca-a501-440e-9b30-2ece477419a4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:42.256271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 323/629] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jul 11 20:44:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:42.257343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d788f39-4300-4281-b99a-0951628cec25 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:42.265520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 307/630] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:44:42 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/97b5f28d84fe4570a7c5c904dc4e502c/roles/91ff1620e1064bf199221383f124aa3c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:42.267984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aee89-2741-48ac-bba4-997eb024b738 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.268532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aee89-2741-48ac-bba4-997eb024b738 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.269035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aee89-2741-48ac-bba4-997eb024b738 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.271913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 308/631] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:42.273161 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92aee89-2741-48ac-bba4-997eb024b738 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:42.338563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:42.387850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d788f39-4300-4281-b99a-0951628cec25 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:42.415958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92aee89-2741-48ac-bba4-997eb024b738 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:42.432543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396ac3e-9039-4ffe-a0ac-7a4c2de45f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:42.433430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396ac3e-9039-4ffe-a0ac-7a4c2de45f99 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.433718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396ac3e-9039-4ffe-a0ac-7a4c2de45f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.433991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396ac3e-9039-4ffe-a0ac-7a4c2de45f99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.446548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396ac3e-9039-4ffe-a0ac-7a4c2de45f99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a396ac3e-9039-4ffe-a0ac-7a4c2de45f99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:42.455135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:42.456085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:42.456964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 324/632] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:42.458199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f528-0be1-48d4-b3e9-ea541c8e98da None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.458539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f528-0be1-48d4-b3e9-ea541c8e98da None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.458820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f528-0be1-48d4-b3e9-ea541c8e98da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.461074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1b412-ef41-44f7-bccb-6855e567c286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:42.461812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1b412-ef41-44f7-bccb-6855e567c286 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.462114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1b412-ef41-44f7-bccb-6855e567c286 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.462375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1b412-ef41-44f7-bccb-6855e567c286 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.477611 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4aa8f528-0be1-48d4-b3e9-ea541c8e98da None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:42.478793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4aa8f528-0be1-48d4-b3e9-ea541c8e98da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:42.546577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:42.548852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1b412-ef41-44f7-bccb-6855e567c286 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:42.550001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1b412-ef41-44f7-bccb-6855e567c286 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:42.578295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 325/633] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:42.579174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8f528-0be1-48d4-b3e9-ea541c8e98da None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:42.596615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:42.662352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 309/634] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:42.663533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1b412-ef41-44f7-bccb-6855e567c286 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:42.689016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8f528-0be1-48d4-b3e9-ea541c8e98da None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:42.726857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b2ebc-cc7b-4325-8d28-77e3fcc1e16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:42.727538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2ebc-cc7b-4325-8d28-77e3fcc1e16d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.727820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2ebc-cc7b-4325-8d28-77e3fcc1e16d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.728094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2ebc-cc7b-4325-8d28-77e3fcc1e16d None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.728877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b2ebc-cc7b-4325-8d28-77e3fcc1e16d None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:42.729252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b2ebc-cc7b-4325-8d28-77e3fcc1e16d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:42.730564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b2ebc-cc7b-4325-8d28-77e3fcc1e16d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:42.739132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 326/635] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:42.740173 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b2ebc-cc7b-4325-8d28-77e3fcc1e16d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:42.775275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1b412-ef41-44f7-bccb-6855e567c286 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:42.792662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:42.811633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e050200-ce57-4f81-af4d-ccff5e2638f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:42.812479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e050200-ce57-4f81-af4d-ccff5e2638f1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.812768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e050200-ce57-4f81-af4d-ccff5e2638f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.813030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e050200-ce57-4f81-af4d-ccff5e2638f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.823070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:42.826513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e050200-ce57-4f81-af4d-ccff5e2638f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-4e050200-ce57-4f81-af4d-ccff5e2638f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:42.835453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:42.836670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:42.837500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 310/636] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:42.838973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e47fe7-9ffd-4461-8c96-2df8e25ca6c7 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.838973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e47fe7-9ffd-4461-8c96-2df8e25ca6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.838973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e47fe7-9ffd-4461-8c96-2df8e25ca6c7 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.840091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 311/637] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:42.844746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e47fe7-9ffd-4461-8c96-2df8e25ca6c7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:42.852306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b2ebc-cc7b-4325-8d28-77e3fcc1e16d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:42.887696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cc976-d754-406e-a365-415f3ac57ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:42.888530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cc976-d754-406e-a365-415f3ac57ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.888813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cc976-d754-406e-a365-415f3ac57ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:42.889086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cc976-d754-406e-a365-415f3ac57ec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:42.900385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cc976-d754-406e-a365-415f3ac57ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:42.901830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cc976-d754-406e-a365-415f3ac57ec1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-cb1cc976-d754-406e-a365-415f3ac57ec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:42.918894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:42.919809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:42.920712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:42.920712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:42.920712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:42.920712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:42.920712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 327/638] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:42.921417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cc976-d754-406e-a365-415f3ac57ec1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:42.962000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e47fe7-9ffd-4461-8c96-2df8e25ca6c7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:42.998868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec10b56-7d0a-43a1-9365-f2a33a88bd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:42.999677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec10b56-7d0a-43a1-9365-f2a33a88bd0f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:42.999955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec10b56-7d0a-43a1-9365-f2a33a88bd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.000256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec10b56-7d0a-43a1-9365-f2a33a88bd0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.012336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec10b56-7d0a-43a1-9365-f2a33a88bd0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.013967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec10b56-7d0a-43a1-9365-f2a33a88bd0f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:43.023294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cc976-d754-406e-a365-415f3ac57ec1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.024847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-bec10b56-7d0a-43a1-9365-f2a33a88bd0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:43.032987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:43.033953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:43.034852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:43.034852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:43.034852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:43.034852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:43.034852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 312/639] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:43.035543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9299-9a56-4c15-b203-904ad6505951 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.035792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9299-9a56-4c15-b203-904ad6505951 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.036066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9299-9a56-4c15-b203-904ad6505951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.054106 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-db9e9299-9a56-4c15-b203-904ad6505951 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:43.055301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-db9e9299-9a56-4c15-b203-904ad6505951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:43.066296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090b0c8-9484-4a42-bd37-75344b068595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.067043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090b0c8-9484-4a42-bd37-75344b068595 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.067332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090b0c8-9484-4a42-bd37-75344b068595 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.067606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090b0c8-9484-4a42-bd37-75344b068595 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.068640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090b0c8-9484-4a42-bd37-75344b068595 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:43.068944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090b0c8-9484-4a42-bd37-75344b068595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.070355 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090b0c8-9484-4a42-bd37-75344b068595 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:43.079065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 328/640] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:43.079676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090b0c8-9484-4a42-bd37-75344b068595 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.109907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:43.136148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 313/641] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:43.136148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e9299-9a56-4c15-b203-904ad6505951 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.195383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090b0c8-9484-4a42-bd37-75344b068595 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.228986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db90f0-51b1-499f-8d0f-6d9704e7ddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.229653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db90f0-51b1-499f-8d0f-6d9704e7ddba None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.229939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db90f0-51b1-499f-8d0f-6d9704e7ddba None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.230214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db90f0-51b1-499f-8d0f-6d9704e7ddba None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.230946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db90f0-51b1-499f-8d0f-6d9704e7ddba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:43.231251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db90f0-51b1-499f-8d0f-6d9704e7ddba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.232685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db90f0-51b1-499f-8d0f-6d9704e7ddba None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:43.244565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 329/642] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:43.245399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db90f0-51b1-499f-8d0f-6d9704e7ddba None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.255472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e9299-9a56-4c15-b203-904ad6505951 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.273118 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:43.292435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bccc0-34c5-4423-b473-7b17f8069ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.293121 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bccc0-34c5-4423-b473-7b17f8069ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.293393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bccc0-34c5-4423-b473-7b17f8069ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.293978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bccc0-34c5-4423-b473-7b17f8069ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.294726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bccc0-34c5-4423-b473-7b17f8069ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:43.295033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bccc0-34c5-4423-b473-7b17f8069ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.296444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bccc0-34c5-4423-b473-7b17f8069ad3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:43.309196 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 314/643] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:43.309817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bccc0-34c5-4423-b473-7b17f8069ad3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.357369 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db90f0-51b1-499f-8d0f-6d9704e7ddba None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.358404 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:43.391695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c614516-cd9f-43da-a95b-598584e9993e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.392514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c614516-cd9f-43da-a95b-598584e9993e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.392796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c614516-cd9f-43da-a95b-598584e9993e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.393066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c614516-cd9f-43da-a95b-598584e9993e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.404032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c614516-cd9f-43da-a95b-598584e9993e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.405537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c614516-cd9f-43da-a95b-598584e9993e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:43.421135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bccc0-34c5-4423-b473-7b17f8069ad3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6c614516-cd9f-43da-a95b-598584e9993e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:43.423194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:43.424167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:43.425058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:43.425058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:43.425058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:43.425058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:43.425058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 330/644] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:43.425339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c614516-cd9f-43da-a95b-598584e9993e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.463705 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c7479-a9c0-4f04-aeaa-9ad5f2e2093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.465035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c7479-a9c0-4f04-aeaa-9ad5f2e2093e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.465515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c7479-a9c0-4f04-aeaa-9ad5f2e2093e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.465982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c7479-a9c0-4f04-aeaa-9ad5f2e2093e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.478835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c7479-a9c0-4f04-aeaa-9ad5f2e2093e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-319c7479-a9c0-4f04-aeaa-9ad5f2e2093e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:43.486759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:43.487665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 315/645] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:43.489024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c7479-a9c0-4f04-aeaa-9ad5f2e2093e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.517510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:43.545630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c614516-cd9f-43da-a95b-598584e9993e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.582445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e638d5d-f5c1-4da0-9bda-c442f6e1782d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.584095 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e638d5d-f5c1-4da0-9bda-c442f6e1782d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.584608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e638d5d-f5c1-4da0-9bda-c442f6e1782d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.585085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e638d5d-f5c1-4da0-9bda-c442f6e1782d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.592128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e638d5d-f5c1-4da0-9bda-c442f6e1782d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.593801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e638d5d-f5c1-4da0-9bda-c442f6e1782d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:43.594083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c7479-a9c0-4f04-aeaa-9ad5f2e2093e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.595736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-0e638d5d-f5c1-4da0-9bda-c442f6e1782d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:43.612967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:43.614033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:43.615191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:43.615191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:43.615191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:43.615191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:43.615191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:43.615191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:43.615191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:43.615191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:43.615191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 331/646] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:43.615696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e638d5d-f5c1-4da0-9bda-c442f6e1782d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.635306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c31cb-2793-4820-b0b0-b79d22e1a61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.636157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c31cb-2793-4820-b0b0-b79d22e1a61d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.636430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c31cb-2793-4820-b0b0-b79d22e1a61d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.636704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c31cb-2793-4820-b0b0-b79d22e1a61d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.644568 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c31cb-2793-4820-b0b0-b79d22e1a61d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.646071 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c31cb-2793-4820-b0b0-b79d22e1a61d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-506c31cb-2793-4820-b0b0-b79d22e1a61d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:43.660461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:43.661419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:43.662380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 316/647] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:43.662981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c31cb-2793-4820-b0b0-b79d22e1a61d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.725826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e638d5d-f5c1-4da0-9bda-c442f6e1782d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.759452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a70d7-8b29-49a9-aa9d-b469f1db81cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.760133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a70d7-8b29-49a9-aa9d-b469f1db81cf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.760441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a70d7-8b29-49a9-aa9d-b469f1db81cf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.760713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a70d7-8b29-49a9-aa9d-b469f1db81cf None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.761427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a70d7-8b29-49a9-aa9d-b469f1db81cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:43.761727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a70d7-8b29-49a9-aa9d-b469f1db81cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.763103 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a70d7-8b29-49a9-aa9d-b469f1db81cf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:43.765649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:43.773539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c31cb-2793-4820-b0b0-b79d22e1a61d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.777073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 332/648] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:43.778074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a70d7-8b29-49a9-aa9d-b469f1db81cf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.812771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125da3a0-c0b9-4768-b037-0b0f3c2a11ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.813451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125da3a0-c0b9-4768-b037-0b0f3c2a11ee None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.813732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125da3a0-c0b9-4768-b037-0b0f3c2a11ee None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.814005 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125da3a0-c0b9-4768-b037-0b0f3c2a11ee None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.814771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125da3a0-c0b9-4768-b037-0b0f3c2a11ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:43.815073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125da3a0-c0b9-4768-b037-0b0f3c2a11ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.816485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125da3a0-c0b9-4768-b037-0b0f3c2a11ee None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:43.825571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 317/649] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:43.826389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125da3a0-c0b9-4768-b037-0b0f3c2a11ee None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.857589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:43.899042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a70d7-8b29-49a9-aa9d-b469f1db81cf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.934308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd706e-d59b-4b32-b131-82dba1b3e07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.935847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd706e-d59b-4b32-b131-82dba1b3e07a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.936149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd706e-d59b-4b32-b131-82dba1b3e07a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.936433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd706e-d59b-4b32-b131-82dba1b3e07a None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.937229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd706e-d59b-4b32-b131-82dba1b3e07a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:43.937542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd706e-d59b-4b32-b131-82dba1b3e07a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.938985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd706e-d59b-4b32-b131-82dba1b3e07a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:43.940431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125da3a0-c0b9-4768-b037-0b0f3c2a11ee None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:43.951629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 333/650] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:43.952551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd706e-d59b-4b32-b131-82dba1b3e07a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:43.983022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0aea8-bfc1-4171-9e1b-73e79d1f4e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:43.983891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0aea8-bfc1-4171-9e1b-73e79d1f4e3d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:43.984512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0aea8-bfc1-4171-9e1b-73e79d1f4e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:43.984795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0aea8-bfc1-4171-9e1b-73e79d1f4e3d None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:43.985800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0aea8-bfc1-4171-9e1b-73e79d1f4e3d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:43.986120 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0aea8-bfc1-4171-9e1b-73e79d1f4e3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:43.987555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0aea8-bfc1-4171-9e1b-73e79d1f4e3d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:44.005352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 318/651] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:44.009986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0aea8-bfc1-4171-9e1b-73e79d1f4e3d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:44.017526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:44.071729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd706e-d59b-4b32-b131-82dba1b3e07a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:44.091400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:44.109878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6ca5-50b7-4071-a10d-3e3604118234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:44.110286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0aea8-bfc1-4171-9e1b-73e79d1f4e3d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:44.110691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6ca5-50b7-4071-a10d-3e3604118234 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:44.110980 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6ca5-50b7-4071-a10d-3e3604118234 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:44.111256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6ca5-50b7-4071-a10d-3e3604118234 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:44.120238 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be6ca5-50b7-4071-a10d-3e3604118234 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:44.121712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be6ca5-50b7-4071-a10d-3e3604118234 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-41be6ca5-50b7-4071-a10d-3e3604118234 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:44.136093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:44.137139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 334/652] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jul 11 20:44:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:44.138072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6ca5-50b7-4071-a10d-3e3604118234 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:44.150941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b533e-41db-4ca2-aac0-acb9deee295e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:44.151680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b533e-41db-4ca2-aac0-acb9deee295e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:44.151962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b533e-41db-4ca2-aac0-acb9deee295e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:44.152278 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b533e-41db-4ca2-aac0-acb9deee295e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:44.162831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b533e-41db-4ca2-aac0-acb9deee295e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:44.164342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b533e-41db-4ca2-aac0-acb9deee295e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-397b533e-41db-4ca2-aac0-acb9deee295e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:44.181919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:44.182903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:44.183790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:44.183790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:44.183790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:44.183790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:44.183790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 319/653] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:44.184076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b533e-41db-4ca2-aac0-acb9deee295e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:44.246995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6ca5-50b7-4071-a10d-3e3604118234 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:44.263340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:44.283009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75610c60-6360-4987-950e-891b658e5c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:44.283674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75610c60-6360-4987-950e-891b658e5c06 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:44.283954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75610c60-6360-4987-950e-891b658e5c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:44.284274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75610c60-6360-4987-950e-891b658e5c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:44.285113 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75610c60-6360-4987-950e-891b658e5c06 None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=097b12fbf6494336b11ece5c498a4103) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:44.285424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75610c60-6360-4987-950e-891b658e5c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:44.286982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75610c60-6360-4987-950e-891b658e5c06 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:44.293240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 335/654] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:44 2022] GET /identity/v3/role_assignments?user.id=097b12fbf6494336b11ece5c498a4103&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:44.304500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75610c60-6360-4987-950e-891b658e5c06 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:44.311400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b533e-41db-4ca2-aac0-acb9deee295e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:44.362505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452d5c8-f509-4b13-bafa-7dec181c469e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:44.363295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d5c8-f509-4b13-bafa-7dec181c469e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:44.363689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d5c8-f509-4b13-bafa-7dec181c469e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:44.364033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d5c8-f509-4b13-bafa-7dec181c469e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:44.368893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:44.428132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75610c60-6360-4987-950e-891b658e5c06 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:44.451287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452d5c8-f509-4b13-bafa-7dec181c469e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:44.452824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452d5c8-f509-4b13-bafa-7dec181c469e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:44.471116 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e25c2f-87dd-48eb-bda7-fba48c4f790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:44.471701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e25c2f-87dd-48eb-bda7-fba48c4f790c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:44.471980 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e25c2f-87dd-48eb-bda7-fba48c4f790c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:44.473520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e25c2f-87dd-48eb-bda7-fba48c4f790c None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:44.473520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e25c2f-87dd-48eb-bda7-fba48c4f790c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:44.473520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e25c2f-87dd-48eb-bda7-fba48c4f790c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:44.474924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e25c2f-87dd-48eb-bda7-fba48c4f790c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:44.495061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 336/655] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jul 11 20:44:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:44.495914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e25c2f-87dd-48eb-bda7-fba48c4f790c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:44.540069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:44.569616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 320/656] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:44.571123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6adde4-dfdd-40ab-8f79-68fd9aeffe4e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:44.571504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6adde4-dfdd-40ab-8f79-68fd9aeffe4e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:44.571870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6adde4-dfdd-40ab-8f79-68fd9aeffe4e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:44.573621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 321/657] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:44.574947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6adde4-dfdd-40ab-8f79-68fd9aeffe4e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:44.612824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e25c2f-87dd-48eb-bda7-fba48c4f790c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:44.613364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:44.651041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10b9bb-bddb-410c-b33d-4e7f20943fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:44.651716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10b9bb-bddb-410c-b33d-4e7f20943fdf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:44.651995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10b9bb-bddb-410c-b33d-4e7f20943fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:44.652308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10b9bb-bddb-410c-b33d-4e7f20943fdf None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:44.653038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10b9bb-bddb-410c-b33d-4e7f20943fdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:44.653344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10b9bb-bddb-410c-b33d-4e7f20943fdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:44.654721 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10b9bb-bddb-410c-b33d-4e7f20943fdf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:44.666838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 337/658] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:44.667704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10b9bb-bddb-410c-b33d-4e7f20943fdf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:44.694493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6adde4-dfdd-40ab-8f79-68fd9aeffe4e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:44.731217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffecd13-9436-4d0b-a82f-27b9455da82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:44.731994 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffecd13-9436-4d0b-a82f-27b9455da82b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:44.732361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffecd13-9436-4d0b-a82f-27b9455da82b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:44.732703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffecd13-9436-4d0b-a82f-27b9455da82b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:44.772620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:44.777359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10b9bb-bddb-410c-b33d-4e7f20943fdf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:44.806657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffecd13-9436-4d0b-a82f-27b9455da82b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:44.808224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffecd13-9436-4d0b-a82f-27b9455da82b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:44.819329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eb4ac-aedf-44af-a70b-fbdaa85e7245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:44.820821 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb4ac-aedf-44af-a70b-fbdaa85e7245 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:44.821112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb4ac-aedf-44af-a70b-fbdaa85e7245 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:44.821390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb4ac-aedf-44af-a70b-fbdaa85e7245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:44.837938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eb4ac-aedf-44af-a70b-fbdaa85e7245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:44.839716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eb4ac-aedf-44af-a70b-fbdaa85e7245 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:44.843033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-847eb4ac-aedf-44af-a70b-fbdaa85e7245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:44.859500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:44.860876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:44.861970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:44.861970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:44.861970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:44.861970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:44.866216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 338/659] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:44.867109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f299bae-c2c2-4c02-8b66-47e0c8068d23 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:44.867398 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f299bae-c2c2-4c02-8b66-47e0c8068d23 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:44.867743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f299bae-c2c2-4c02-8b66-47e0c8068d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:44.884799 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4f299bae-c2c2-4c02-8b66-47e0c8068d23 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:44.885978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4f299bae-c2c2-4c02-8b66-47e0c8068d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:44.920332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 322/660] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:44.921401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffecd13-9436-4d0b-a82f-27b9455da82b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:44.961026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 339/661] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:44.962019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f299bae-c2c2-4c02-8b66-47e0c8068d23 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.038346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffecd13-9436-4d0b-a82f-27b9455da82b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.055762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:45.072033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580545a3-f559-47a7-b4fa-99023c1e079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.072861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580545a3-f559-47a7-b4fa-99023c1e079e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.073141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580545a3-f559-47a7-b4fa-99023c1e079e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.073412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580545a3-f559-47a7-b4fa-99023c1e079e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.075717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:45.079092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f299bae-c2c2-4c02-8b66-47e0c8068d23 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.081876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580545a3-f559-47a7-b4fa-99023c1e079e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:45.083582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580545a3-f559-47a7-b4fa-99023c1e079e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-580545a3-f559-47a7-b4fa-99023c1e079e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:45.097988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:45.099399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 323/662] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:45.100356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580545a3-f559-47a7-b4fa-99023c1e079e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.119989 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a122d31-1f94-4aa4-b802-d73c23fcc743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.120942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a122d31-1f94-4aa4-b802-d73c23fcc743 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.121376 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a122d31-1f94-4aa4-b802-d73c23fcc743 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.121900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a122d31-1f94-4aa4-b802-d73c23fcc743 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.122896 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a122d31-1f94-4aa4-b802-d73c23fcc743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:45.123340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a122d31-1f94-4aa4-b802-d73c23fcc743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:45.124878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a122d31-1f94-4aa4-b802-d73c23fcc743 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:45.137602 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 340/663] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:45.138727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a122d31-1f94-4aa4-b802-d73c23fcc743 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.211839 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580545a3-f559-47a7-b4fa-99023c1e079e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.241556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a122d31-1f94-4aa4-b802-d73c23fcc743 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.246680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24deb80-feb2-4f62-a4a8-5d0ab7d993e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.247485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24deb80-feb2-4f62-a4a8-5d0ab7d993e8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.247766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24deb80-feb2-4f62-a4a8-5d0ab7d993e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.248044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24deb80-feb2-4f62-a4a8-5d0ab7d993e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.259059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24deb80-feb2-4f62-a4a8-5d0ab7d993e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:45.260541 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24deb80-feb2-4f62-a4a8-5d0ab7d993e8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:45.275008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530423fa-1076-436a-a656-b9d4987ab8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.275817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530423fa-1076-436a-a656-b9d4987ab8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.276092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530423fa-1076-436a-a656-b9d4987ab8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.276394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530423fa-1076-436a-a656-b9d4987ab8e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a24deb80-feb2-4f62-a4a8-5d0ab7d993e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:45.278221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:45.279045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:45.280730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:45.280730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:45.280730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:45.280730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:45.280730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 324/664] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:45.280730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24deb80-feb2-4f62-a4a8-5d0ab7d993e8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.289921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:45.292881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530423fa-1076-436a-a656-b9d4987ab8e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-530423fa-1076-436a-a656-b9d4987ab8e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:45.301788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:45.302934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:45.303755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:45.306850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 341/665] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:45.312473 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530423fa-1076-436a-a656-b9d4987ab8e4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.331155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:45.385095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24deb80-feb2-4f62-a4a8-5d0ab7d993e8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.418453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530423fa-1076-436a-a656-b9d4987ab8e4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.420694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4460b59-c335-47a3-bc75-6081908a21cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.421387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4460b59-c335-47a3-bc75-6081908a21cd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.421666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4460b59-c335-47a3-bc75-6081908a21cd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.421939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4460b59-c335-47a3-bc75-6081908a21cd None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.422791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4460b59-c335-47a3-bc75-6081908a21cd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:45.423100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4460b59-c335-47a3-bc75-6081908a21cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:45.424585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4460b59-c335-47a3-bc75-6081908a21cd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:45.446298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 325/666] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:45.447153 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4460b59-c335-47a3-bc75-6081908a21cd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.462575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fce81f-232b-437c-8847-a5c0b4b554e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.463360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fce81f-232b-437c-8847-a5c0b4b554e5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.463687 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fce81f-232b-437c-8847-a5c0b4b554e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.463961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fce81f-232b-437c-8847-a5c0b4b554e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.472088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fce81f-232b-437c-8847-a5c0b4b554e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:45.473822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fce81f-232b-437c-8847-a5c0b4b554e5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-28fce81f-232b-437c-8847-a5c0b4b554e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:45.491048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:45.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:45.492890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:45.492890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:45.492890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:45.492890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:45.492890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:45.492890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:45.492890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:45.492890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:45.492890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 342/667] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:45.493361 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fce81f-232b-437c-8847-a5c0b4b554e5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.553658 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:45.563632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4460b59-c335-47a3-bc75-6081908a21cd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.567047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:45.595464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fce81f-232b-437c-8847-a5c0b4b554e5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.602274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139e77e-126c-41e7-a4c4-fae17d6d51d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.603063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e77e-126c-41e7-a4c4-fae17d6d51d5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.603349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e77e-126c-41e7-a4c4-fae17d6d51d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.603620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e77e-126c-41e7-a4c4-fae17d6d51d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.604378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139e77e-126c-41e7-a4c4-fae17d6d51d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:45.604685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139e77e-126c-41e7-a4c4-fae17d6d51d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:45.606156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139e77e-126c-41e7-a4c4-fae17d6d51d5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:45.622783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 326/668] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:45.622783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139e77e-126c-41e7-a4c4-fae17d6d51d5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.634286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58596c-11b9-4dac-bef6-f9adeee535f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.635001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58596c-11b9-4dac-bef6-f9adeee535f7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.635236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58596c-11b9-4dac-bef6-f9adeee535f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.635504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58596c-11b9-4dac-bef6-f9adeee535f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.636297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58596c-11b9-4dac-bef6-f9adeee535f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:45.636598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58596c-11b9-4dac-bef6-f9adeee535f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:45.637964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58596c-11b9-4dac-bef6-f9adeee535f7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:45.650411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 343/669] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:45.650411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58596c-11b9-4dac-bef6-f9adeee535f7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.730975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139e77e-126c-41e7-a4c4-fae17d6d51d5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.749386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58596c-11b9-4dac-bef6-f9adeee535f7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.766576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dbfbf-bd71-4f10-9eb6-9b7248e1db68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.767266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dbfbf-bd71-4f10-9eb6-9b7248e1db68 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.767545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dbfbf-bd71-4f10-9eb6-9b7248e1db68 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.767812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dbfbf-bd71-4f10-9eb6-9b7248e1db68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.774593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:45.790495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c0296-aafe-4134-887a-80ee6a22ee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.791568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c0296-aafe-4134-887a-80ee6a22ee4b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.791865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c0296-aafe-4134-887a-80ee6a22ee4b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.792172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c0296-aafe-4134-887a-80ee6a22ee4b None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.792966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c0296-aafe-4134-887a-80ee6a22ee4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:45.793274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c0296-aafe-4134-887a-80ee6a22ee4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:45.794748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c0296-aafe-4134-887a-80ee6a22ee4b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:45.801003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:45.809629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 344/670] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:45.815021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c0296-aafe-4134-887a-80ee6a22ee4b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.845900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dbfbf-bd71-4f10-9eb6-9b7248e1db68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:45.847374 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dbfbf-bd71-4f10-9eb6-9b7248e1db68 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:45.925351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c0296-aafe-4134-887a-80ee6a22ee4b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:45.962192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 327/671] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:45.962192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dbfbf-bd71-4f10-9eb6-9b7248e1db68 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:45.968863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d88f89-a208-477b-be1e-c87ace4a8e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:45.970026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d88f89-a208-477b-be1e-c87ace4a8e34 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:45.970306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d88f89-a208-477b-be1e-c87ace4a8e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:45.970576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d88f89-a208-477b-be1e-c87ace4a8e34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:45.977926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d88f89-a208-477b-be1e-c87ace4a8e34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:45.979377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d88f89-a208-477b-be1e-c87ace4a8e34 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-94d88f89-a208-477b-be1e-c87ace4a8e34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:45.993333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:45.994408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:45.995570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:45.995570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:45.995570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:45.995570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:45.995570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:45.995570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:45.995570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:45.995570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:45.999174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 345/672] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:45.999871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d88f89-a208-477b-be1e-c87ace4a8e34 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:46.026634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:46.048572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:46.065895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dbfbf-bd71-4f10-9eb6-9b7248e1db68 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:46.104281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582633ff-c992-4b5e-ad4a-985217592161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:46.105093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582633ff-c992-4b5e-ad4a-985217592161 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.105377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582633ff-c992-4b5e-ad4a-985217592161 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.105654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582633ff-c992-4b5e-ad4a-985217592161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.116862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d88f89-a208-477b-be1e-c87ace4a8e34 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:46.117132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582633ff-c992-4b5e-ad4a-985217592161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:46.118714 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582633ff-c992-4b5e-ad4a-985217592161 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-582633ff-c992-4b5e-ad4a-985217592161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:46.138180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:46.139110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:46.139991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:46.139991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:46.139991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:46.139991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:46.139991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 328/673] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:46.140728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582633ff-c992-4b5e-ad4a-985217592161 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:46.156930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33768de-43ad-4768-8ec7-880a6edf640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:46.157619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33768de-43ad-4768-8ec7-880a6edf640e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.157886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33768de-43ad-4768-8ec7-880a6edf640e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.158160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33768de-43ad-4768-8ec7-880a6edf640e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.159004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33768de-43ad-4768-8ec7-880a6edf640e None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=dee928cd256446cb9e000ceb6e9ef636) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:46.159317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33768de-43ad-4768-8ec7-880a6edf640e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:46.160864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33768de-43ad-4768-8ec7-880a6edf640e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:46.167023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 346/674] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:45 2022] GET /identity/v3/role_assignments?user.id=dee928cd256446cb9e000ceb6e9ef636&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:46.177585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33768de-43ad-4768-8ec7-880a6edf640e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:46.259197 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582633ff-c992-4b5e-ad4a-985217592161 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:46.278017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:46.298809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1566e-4580-4512-8f29-00e19a18b537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:46.299626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1566e-4580-4512-8f29-00e19a18b537 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.299904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1566e-4580-4512-8f29-00e19a18b537 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.300219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1566e-4580-4512-8f29-00e19a18b537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.311771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1566e-4580-4512-8f29-00e19a18b537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:46.313289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1566e-4580-4512-8f29-00e19a18b537 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:46.331720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33768de-43ad-4768-8ec7-880a6edf640e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-99d1566e-4580-4512-8f29-00e19a18b537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:46.332487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:46.334325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:46.335215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:46.335215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:46.335215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:46.335215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:46.335215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 329/675] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:46.335215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1566e-4580-4512-8f29-00e19a18b537 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:46.342884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:46.373484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fe061-31d5-4e7b-9ae7-bc35fa944f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:46.374151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fe061-31d5-4e7b-9ae7-bc35fa944f4e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.374431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fe061-31d5-4e7b-9ae7-bc35fa944f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.374793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fe061-31d5-4e7b-9ae7-bc35fa944f4e None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.375487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fe061-31d5-4e7b-9ae7-bc35fa944f4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:46.375794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fe061-31d5-4e7b-9ae7-bc35fa944f4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:46.377271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fe061-31d5-4e7b-9ae7-bc35fa944f4e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:46.396139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 347/676] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:46.396139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fe061-31d5-4e7b-9ae7-bc35fa944f4e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:46.450072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1566e-4580-4512-8f29-00e19a18b537 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:46.485090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edce96d-a958-43aa-86c2-fd820fe710b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:46.485765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edce96d-a958-43aa-86c2-fd820fe710b1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.486047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edce96d-a958-43aa-86c2-fd820fe710b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.486386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edce96d-a958-43aa-86c2-fd820fe710b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.487042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edce96d-a958-43aa-86c2-fd820fe710b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:46.487344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edce96d-a958-43aa-86c2-fd820fe710b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:46.488776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edce96d-a958-43aa-86c2-fd820fe710b1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:46.499176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fe061-31d5-4e7b-9ae7-bc35fa944f4e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:46.504327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 330/677] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:46.504327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef9b45-d223-4a06-8e98-3e6e7d6f2ec9 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.504327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef9b45-d223-4a06-8e98-3e6e7d6f2ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.504327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef9b45-d223-4a06-8e98-3e6e7d6f2ec9 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.504327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 331/678] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:46.504327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef9b45-d223-4a06-8e98-3e6e7d6f2ec9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:46.533912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:46.542952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef01b2-7e0c-43f4-a10f-a8df14c98b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:46.543633 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef01b2-7e0c-43f4-a10f-a8df14c98b83 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.543919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef01b2-7e0c-43f4-a10f-a8df14c98b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.544226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef01b2-7e0c-43f4-a10f-a8df14c98b83 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.544963 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef01b2-7e0c-43f4-a10f-a8df14c98b83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:46.545298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef01b2-7e0c-43f4-a10f-a8df14c98b83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:46.546680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef01b2-7e0c-43f4-a10f-a8df14c98b83 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:46.558225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 348/679] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:46.559054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef01b2-7e0c-43f4-a10f-a8df14c98b83 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:46.589362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:46.621171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef9b45-d223-4a06-8e98-3e6e7d6f2ec9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:46.656120 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b336d-3574-419f-aede-df9e387af3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:46.656815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b336d-3574-419f-aede-df9e387af3dc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.657095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b336d-3574-419f-aede-df9e387af3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.657373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b336d-3574-419f-aede-df9e387af3dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.658229 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b336d-3574-419f-aede-df9e387af3dc None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=acb3ad858e4848b9a325c72c7328c863) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:46.658534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b336d-3574-419f-aede-df9e387af3dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:46.659972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b336d-3574-419f-aede-df9e387af3dc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:46.672132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 332/680] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:46 2022] GET /identity/v3/role_assignments?user.id=acb3ad858e4848b9a325c72c7328c863&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:46.672132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0168ec-b662-43fd-8778-96c8a7e71abe None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.672132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0168ec-b662-43fd-8778-96c8a7e71abe None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.672516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef01b2-7e0c-43f4-a10f-a8df14c98b83 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:46.672944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0168ec-b662-43fd-8778-96c8a7e71abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.694189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8b0168ec-b662-43fd-8778-96c8a7e71abe None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:46.695429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8b0168ec-b662-43fd-8778-96c8a7e71abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:46.720406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c15240-8c71-417e-87ad-acc55bf9992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:46.721544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c15240-8c71-417e-87ad-acc55bf9992b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.721923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c15240-8c71-417e-87ad-acc55bf9992b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.722290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c15240-8c71-417e-87ad-acc55bf9992b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.735283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c15240-8c71-417e-87ad-acc55bf9992b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:46.737080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c15240-8c71-417e-87ad-acc55bf9992b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d2c15240-8c71-417e-87ad-acc55bf9992b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:46.757996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:46.759107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:46.760036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:46.760036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:46.760036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:46.760036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:46.760036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:46.760036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:46.760036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:46.760036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:46.760036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:44:46.760036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:46.763859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 349/681] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:46.764903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c15240-8c71-417e-87ad-acc55bf9992b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:46.792650 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 333/682] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:46.794059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0168ec-b662-43fd-8778-96c8a7e71abe None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:46.841879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:46.879029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:46.903287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c15240-8c71-417e-87ad-acc55bf9992b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:46.948533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0168ec-b662-43fd-8778-96c8a7e71abe None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:46.949928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c64ac-f793-4f63-aaa9-2cc4b5f43cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:46.950745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c64ac-f793-4f63-aaa9-2cc4b5f43cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.951027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c64ac-f793-4f63-aaa9-2cc4b5f43cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.951301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c64ac-f793-4f63-aaa9-2cc4b5f43cf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.958856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c64ac-f793-4f63-aaa9-2cc4b5f43cf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:46.960657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c64ac-f793-4f63-aaa9-2cc4b5f43cf9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-864c64ac-f793-4f63-aaa9-2cc4b5f43cf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:46.977162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:46.978134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:46.979002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:46.979002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:46.979002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:46.979002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:46.979002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:46.979002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:46.979002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:46.979002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:46.979002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 350/683] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:46.979498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c64ac-f793-4f63-aaa9-2cc4b5f43cf9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:46.992377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a42d71-5a34-4b07-a024-7e2ef75cbc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:46.993249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a42d71-5a34-4b07-a024-7e2ef75cbc3a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:46.993359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a42d71-5a34-4b07-a024-7e2ef75cbc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:46.993623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a42d71-5a34-4b07-a024-7e2ef75cbc3a None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:46.994489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a42d71-5a34-4b07-a024-7e2ef75cbc3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:46.994802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a42d71-5a34-4b07-a024-7e2ef75cbc3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:46.996284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a42d71-5a34-4b07-a024-7e2ef75cbc3a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:47.015873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 334/684] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jul 11 20:44:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:47.017108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a42d71-5a34-4b07-a024-7e2ef75cbc3a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:47.099303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c64ac-f793-4f63-aaa9-2cc4b5f43cf9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:47.122248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:47.145163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a42d71-5a34-4b07-a024-7e2ef75cbc3a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:47.146943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc189-3aec-4d72-b786-aad3b6698c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:47.147855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc189-3aec-4d72-b786-aad3b6698c2d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:47.148151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc189-3aec-4d72-b786-aad3b6698c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:47.148425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc189-3aec-4d72-b786-aad3b6698c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:47.159614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bc189-3aec-4d72-b786-aad3b6698c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:47.161097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bc189-3aec-4d72-b786-aad3b6698c2d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:47.169572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-e32bc189-3aec-4d72-b786-aad3b6698c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:47.182830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:47.183958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:47.185044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:47.185044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:47.185044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:47.185044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:47.188217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 351/685] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:47.189829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc189-3aec-4d72-b786-aad3b6698c2d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:47.193115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee364ca-ae0c-414e-9f4a-edd292f257e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:47.193961 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee364ca-ae0c-414e-9f4a-edd292f257e9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:47.194250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee364ca-ae0c-414e-9f4a-edd292f257e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:47.194527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee364ca-ae0c-414e-9f4a-edd292f257e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:47.196358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee364ca-ae0c-414e-9f4a-edd292f257e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:47.196680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee364ca-ae0c-414e-9f4a-edd292f257e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:47.198154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee364ca-ae0c-414e-9f4a-edd292f257e9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:47.216186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 335/686] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:47.217364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee364ca-ae0c-414e-9f4a-edd292f257e9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:47.303182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc189-3aec-4d72-b786-aad3b6698c2d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:47.344499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a8037-ed70-4de6-b9b3-a3cc6e62ba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:47.345570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a8037-ed70-4de6-b9b3-a3cc6e62ba69 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:47.345743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee364ca-ae0c-414e-9f4a-edd292f257e9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:47.346536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a8037-ed70-4de6-b9b3-a3cc6e62ba69 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:47.346997 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a8037-ed70-4de6-b9b3-a3cc6e62ba69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:47.395118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea9762-ad67-4037-8bce-da6c80af62c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:47.395861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea9762-ad67-4037-8bce-da6c80af62c1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:47.396191 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea9762-ad67-4037-8bce-da6c80af62c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:47.396473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea9762-ad67-4037-8bce-da6c80af62c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:47.397641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea9762-ad67-4037-8bce-da6c80af62c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:47.397958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea9762-ad67-4037-8bce-da6c80af62c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:47.399655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea9762-ad67-4037-8bce-da6c80af62c1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:47.411528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:47.428557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 336/687] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:47.429445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:47.430009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea9762-ad67-4037-8bce-da6c80af62c1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:47.434179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a8037-ed70-4de6-b9b3-a3cc6e62ba69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:47.436451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a8037-ed70-4de6-b9b3-a3cc6e62ba69 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:47.536712 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea9762-ad67-4037-8bce-da6c80af62c1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:47.568534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 352/688] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:47.569745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a8037-ed70-4de6-b9b3-a3cc6e62ba69 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:47.581505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba31b3f-e0ad-45e8-b0e5-1f11c0c87ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:47.582175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba31b3f-e0ad-45e8-b0e5-1f11c0c87ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:47.582454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba31b3f-e0ad-45e8-b0e5-1f11c0c87ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:47.582724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba31b3f-e0ad-45e8-b0e5-1f11c0c87ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:47.584059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba31b3f-e0ad-45e8-b0e5-1f11c0c87ad1 None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=0574d5c8974f43218653208ca72c43bb) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:47.584400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba31b3f-e0ad-45e8-b0e5-1f11c0c87ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:47.586179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba31b3f-e0ad-45e8-b0e5-1f11c0c87ad1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:47.596174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 337/689] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:47 2022] GET /identity/v3/role_assignments?user.id=0574d5c8974f43218653208ca72c43bb&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:47.597267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba31b3f-e0ad-45e8-b0e5-1f11c0c87ad1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:47.704758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:47.766549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:47.774225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a8037-ed70-4de6-b9b3-a3cc6e62ba69 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:47.808776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba31b3f-e0ad-45e8-b0e5-1f11c0c87ad1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:47.848589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434f66-b46f-43b2-8599-b37082a11d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:47.850131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434f66-b46f-43b2-8599-b37082a11d0e None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:47.850453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434f66-b46f-43b2-8599-b37082a11d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:47.850741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434f66-b46f-43b2-8599-b37082a11d0e None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:47.851908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3434f66-b46f-43b2-8599-b37082a11d0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:47.853964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3434f66-b46f-43b2-8599-b37082a11d0e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:47.862299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56462388-4555-43f3-9ba4-68ffddbe894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:47.863405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56462388-4555-43f3-9ba4-68ffddbe894f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:47.863886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56462388-4555-43f3-9ba4-68ffddbe894f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:47.864383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56462388-4555-43f3-9ba4-68ffddbe894f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:47.877905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56462388-4555-43f3-9ba4-68ffddbe894f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:47.879903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56462388-4555-43f3-9ba4-68ffddbe894f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:47.884312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 353/690] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:44:47 2022] POST /identity/v3/users => generated 254 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-56462388-4555-43f3-9ba4-68ffddbe894f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:47.901601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:47.902580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:47.903466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:47.903466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:47.903466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:47.903466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:47.906316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 338/691] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:47.911934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434f66-b46f-43b2-8599-b37082a11d0e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:48.032976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434f66-b46f-43b2-8599-b37082a11d0e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:48.075879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849899d-4df3-4001-8ed2-475738d045b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:48.076988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849899d-4df3-4001-8ed2-475738d045b3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:48.077490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849899d-4df3-4001-8ed2-475738d045b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:48.078037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849899d-4df3-4001-8ed2-475738d045b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:48.079030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849899d-4df3-4001-8ed2-475738d045b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:48.079540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849899d-4df3-4001-8ed2-475738d045b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:48.081182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849899d-4df3-4001-8ed2-475738d045b3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:48.099746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 354/692] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:48.105552 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56462388-4555-43f3-9ba4-68ffddbe894f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:48.248851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56462388-4555-43f3-9ba4-68ffddbe894f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:48.277944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:48.305331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e10d8-6578-4f08-a8b7-fe5cfedfbcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:48.306069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e10d8-6578-4f08-a8b7-fe5cfedfbcfe None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:48.306345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e10d8-6578-4f08-a8b7-fe5cfedfbcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:48.306611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e10d8-6578-4f08-a8b7-fe5cfedfbcfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:48.313964 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e10d8-6578-4f08-a8b7-fe5cfedfbcfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:48.315405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e10d8-6578-4f08-a8b7-fe5cfedfbcfe None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d89e10d8-6578-4f08-a8b7-fe5cfedfbcfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:48.329668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:48.330645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:48.331533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:48.331533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:48.331533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:48.331533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:48.331533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:48.331533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:48.331533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:48.331533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:48.331533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 339/693] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:48.336666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849899d-4df3-4001-8ed2-475738d045b3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:48.359664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983a50f-bb82-4fcc-8f6b-11b6ea5717fb None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:48.360205 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983a50f-bb82-4fcc-8f6b-11b6ea5717fb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:48.360283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983a50f-bb82-4fcc-8f6b-11b6ea5717fb None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:48.361877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 340/694] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:48.367076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bce27-46e5-44b7-9e9b-4c7b48f46bb9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:48.367362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bce27-46e5-44b7-9e9b-4c7b48f46bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:48.367641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bce27-46e5-44b7-9e9b-4c7b48f46bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:48.374941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:48.386968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d56bce27-46e5-44b7-9e9b-4c7b48f46bb9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:48.388283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d56bce27-46e5-44b7-9e9b-4c7b48f46bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:48.512297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 341/695] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:48.535707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849899d-4df3-4001-8ed2-475738d045b3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:48.553815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bce27-46e5-44b7-9e9b-4c7b48f46bb9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:48.578012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74da38-55c2-4367-8df5-c5b2fdadb0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:48.579060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74da38-55c2-4367-8df5-c5b2fdadb0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:48.579541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74da38-55c2-4367-8df5-c5b2fdadb0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:48.580022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74da38-55c2-4367-8df5-c5b2fdadb0a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:48.582162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74da38-55c2-4367-8df5-c5b2fdadb0a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:48.582677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74da38-55c2-4367-8df5-c5b2fdadb0a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:48.585289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74da38-55c2-4367-8df5-c5b2fdadb0a6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:48.614464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 355/696] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:48.620437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74da38-55c2-4367-8df5-c5b2fdadb0a6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:48.684294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bce27-46e5-44b7-9e9b-4c7b48f46bb9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:48.692162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:48.749079 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e99681-aa6d-4ceb-a7ca-d460422d16d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:48.749922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e99681-aa6d-4ceb-a7ca-d460422d16d3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:48.750206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e99681-aa6d-4ceb-a7ca-d460422d16d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:48.750471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e99681-aa6d-4ceb-a7ca-d460422d16d3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:48.751136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e99681-aa6d-4ceb-a7ca-d460422d16d3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:48.752560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e99681-aa6d-4ceb-a7ca-d460422d16d3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d2e99681-aa6d-4ceb-a7ca-d460422d16d3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:48.791599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:48.792668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:48.794415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 342/697] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Jul 11 20:44:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:48.796309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:48.808827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e99681-aa6d-4ceb-a7ca-d460422d16d3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:48.839719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74da38-55c2-4367-8df5-c5b2fdadb0a6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:48.879646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dd16d-91fd-41f9-838c-d87c8bdd989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:48.879646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd16d-91fd-41f9-838c-d87c8bdd989a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:48.879646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd16d-91fd-41f9-838c-d87c8bdd989a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:48.879646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd16d-91fd-41f9-838c-d87c8bdd989a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:48.880086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5dd16d-91fd-41f9-838c-d87c8bdd989a None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=97b5f28d84fe4570a7c5c904dc4e502c) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:48.880086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5dd16d-91fd-41f9-838c-d87c8bdd989a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:48.880086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5dd16d-91fd-41f9-838c-d87c8bdd989a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:48.884681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 356/698] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:48 2022] GET /identity/v3/role_assignments?user.id=97b5f28d84fe4570a7c5c904dc4e502c&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:49.001449 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e99681-aa6d-4ceb-a7ca-d460422d16d3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:49.042010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac038d-2ffc-4d97-a678-825555b9490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:49.042703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac038d-2ffc-4d97-a678-825555b9490b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:49.042983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac038d-2ffc-4d97-a678-825555b9490b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:49.043257 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac038d-2ffc-4d97-a678-825555b9490b None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:49.043992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac038d-2ffc-4d97-a678-825555b9490b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:49.044328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac038d-2ffc-4d97-a678-825555b9490b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:49.045741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac038d-2ffc-4d97-a678-825555b9490b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:49.051301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 343/699] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jul 11 20:44:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:49.056855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dd16d-91fd-41f9-838c-d87c8bdd989a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:49.073242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322e8b3-83e0-4592-821a-b4f99afbf37e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:49.073957 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322e8b3-83e0-4592-821a-b4f99afbf37e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:49.074155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322e8b3-83e0-4592-821a-b4f99afbf37e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:49.075807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 344/700] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:49.081097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42963227-3b94-444f-8ca1-cd3e9bb48a81 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:49.081385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42963227-3b94-444f-8ca1-cd3e9bb48a81 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:49.081670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42963227-3b94-444f-8ca1-cd3e9bb48a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:49.098628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-42963227-3b94-444f-8ca1-cd3e9bb48a81 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:49.099779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-42963227-3b94-444f-8ca1-cd3e9bb48a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:49.127571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:49.188761 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 345/701] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:49.200010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dd16d-91fd-41f9-838c-d87c8bdd989a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:49.213883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42963227-3b94-444f-8ca1-cd3e9bb48a81 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:49.244947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babb3d9-aa1e-474b-9762-ab75ed783dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:49.245639 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babb3d9-aa1e-474b-9762-ab75ed783dff None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:49.245919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babb3d9-aa1e-474b-9762-ab75ed783dff None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:49.246191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babb3d9-aa1e-474b-9762-ab75ed783dff None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:49.247028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9babb3d9-aa1e-474b-9762-ab75ed783dff None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:49.247428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9babb3d9-aa1e-474b-9762-ab75ed783dff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:49.248967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9babb3d9-aa1e-474b-9762-ab75ed783dff None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:49.255805 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 357/702] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jul 11 20:44:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:49.336609 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42963227-3b94-444f-8ca1-cd3e9bb48a81 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:49.376868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf759588-cc9f-4a4a-8c2b-a01d60dd9dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:49.377720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf759588-cc9f-4a4a-8c2b-a01d60dd9dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:49.377993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf759588-cc9f-4a4a-8c2b-a01d60dd9dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:49.378262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf759588-cc9f-4a4a-8c2b-a01d60dd9dd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:49.390715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf759588-cc9f-4a4a-8c2b-a01d60dd9dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-cf759588-cc9f-4a4a-8c2b-a01d60dd9dd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:49.398817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:49.399809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:49.401336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 346/703] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:49.409458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babb3d9-aa1e-474b-9762-ab75ed783dff None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:49.412865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:49.538669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babb3d9-aa1e-474b-9762-ab75ed783dff None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:49.579606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a384950-6a42-40df-be20-62bd8e7c632e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:49.580318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a384950-6a42-40df-be20-62bd8e7c632e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:49.580600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a384950-6a42-40df-be20-62bd8e7c632e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:49.580874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a384950-6a42-40df-be20-62bd8e7c632e None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:49.581634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a384950-6a42-40df-be20-62bd8e7c632e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:49.581939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a384950-6a42-40df-be20-62bd8e7c632e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:49.583622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a384950-6a42-40df-be20-62bd8e7c632e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:49.592875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 358/704] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:49.598009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf759588-cc9f-4a4a-8c2b-a01d60dd9dd9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:49.630539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:49.725700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf759588-cc9f-4a4a-8c2b-a01d60dd9dd9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:49.766200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e78472-b029-42ee-8491-cbdbc1c0aa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:49.767023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e78472-b029-42ee-8491-cbdbc1c0aa6d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:49.767303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e78472-b029-42ee-8491-cbdbc1c0aa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:49.767574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e78472-b029-42ee-8491-cbdbc1c0aa6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:49.780083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e78472-b029-42ee-8491-cbdbc1c0aa6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:49.781604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e78472-b029-42ee-8491-cbdbc1c0aa6d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-69e78472-b029-42ee-8491-cbdbc1c0aa6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:49.812806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:49.813760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:49.815260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:49.815260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:49.815260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:49.815260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:49.815260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 347/705] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:49.820597 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a384950-6a42-40df-be20-62bd8e7c632e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:49.915815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:49.970341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a384950-6a42-40df-be20-62bd8e7c632e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:50.019589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c888fb-6a6a-4007-b0e1-d9347d7c5649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:50.020302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c888fb-6a6a-4007-b0e1-d9347d7c5649 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:50.020588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c888fb-6a6a-4007-b0e1-d9347d7c5649 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:50.020897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c888fb-6a6a-4007-b0e1-d9347d7c5649 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:50.021732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c888fb-6a6a-4007-b0e1-d9347d7c5649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:50.022043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c888fb-6a6a-4007-b0e1-d9347d7c5649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:50.023423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c888fb-6a6a-4007-b0e1-d9347d7c5649 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:50.036008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 359/706] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:50.041214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e78472-b029-42ee-8491-cbdbc1c0aa6d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:50.160238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:50.192276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e78472-b029-42ee-8491-cbdbc1c0aa6d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:50.242825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba44d1-6510-4ad7-a792-18b4a7b1b997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:50.243639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba44d1-6510-4ad7-a792-18b4a7b1b997 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:50.243926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba44d1-6510-4ad7-a792-18b4a7b1b997 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:50.244225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba44d1-6510-4ad7-a792-18b4a7b1b997 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:50.257242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba44d1-6510-4ad7-a792-18b4a7b1b997 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:50.258704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba44d1-6510-4ad7-a792-18b4a7b1b997 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-56ba44d1-6510-4ad7-a792-18b4a7b1b997 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:50.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:50.277387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:50.278319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:50.278319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:50.278319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:50.278319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:50.278319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:50.278319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:50.278319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:50.278319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:50.278319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:44:50.278319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:50.278846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 348/707] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jul 11 20:44:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:50.283307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c888fb-6a6a-4007-b0e1-d9347d7c5649 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:50.360468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b013-92b4-4b34-93af-dfaf8c1e12bd None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:50.360832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b013-92b4-4b34-93af-dfaf8c1e12bd None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:50.361078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b013-92b4-4b34-93af-dfaf8c1e12bd None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:50.362729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 349/708] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:50.381741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf8b0e-bb27-4647-a6f8-f0121a9d5dbb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:50.382384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf8b0e-bb27-4647-a6f8-f0121a9d5dbb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:50.382384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf8b0e-bb27-4647-a6f8-f0121a9d5dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:50.413685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-29cf8b0e-bb27-4647-a6f8-f0121a9d5dbb None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:50.414884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-29cf8b0e-bb27-4647-a6f8-f0121a9d5dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:50.417632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c888fb-6a6a-4007-b0e1-d9347d7c5649 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:50.449945 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:50.455058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdd9b7-2bf0-4528-a42e-373c492a92af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:50.455747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdd9b7-2bf0-4528-a42e-373c492a92af None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:50.456030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdd9b7-2bf0-4528-a42e-373c492a92af None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:50.456333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdd9b7-2bf0-4528-a42e-373c492a92af None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:50.457184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdd9b7-2bf0-4528-a42e-373c492a92af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:50.457582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdd9b7-2bf0-4528-a42e-373c492a92af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:50.459245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdd9b7-2bf0-4528-a42e-373c492a92af None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:50.494829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 360/709] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jul 11 20:44:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:50.503646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdd9b7-2bf0-4528-a42e-373c492a92af None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:50.557330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 350/710] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:50.558749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bc1a5-1fd5-41c9-bd93-2ae0ae940c74 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:50.559039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bc1a5-1fd5-41c9-bd93-2ae0ae940c74 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:50.559333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bc1a5-1fd5-41c9-bd93-2ae0ae940c74 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:50.568387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 351/711] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:50.574710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7af84-7bdc-4951-bad1-508675bd8c43 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:50.574996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7af84-7bdc-4951-bad1-508675bd8c43 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:50.575371 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7af84-7bdc-4951-bad1-508675bd8c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:50.602321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-72c7af84-7bdc-4951-bad1-508675bd8c43 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:50.603616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-72c7af84-7bdc-4951-bad1-508675bd8c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:50.641497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdd9b7-2bf0-4528-a42e-373c492a92af None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:50.671910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:50.680761 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 352/712] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 20:44:50.684880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7af84-7bdc-4951-bad1-508675bd8c43 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:50.691214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf613cd-8ffd-411d-a0d1-7abe8e6a9407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:50.691877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf613cd-8ffd-411d-a0d1-7abe8e6a9407 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:50.692178 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf613cd-8ffd-411d-a0d1-7abe8e6a9407 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:50.692475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf613cd-8ffd-411d-a0d1-7abe8e6a9407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:50.755446 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:50.781682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf613cd-8ffd-411d-a0d1-7abe8e6a9407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:50.783389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf613cd-8ffd-411d-a0d1-7abe8e6a9407 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:50.812643 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7af84-7bdc-4951-bad1-508675bd8c43 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:50.860688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eea17d-5408-458c-b07e-56f1065d54cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:50.861962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eea17d-5408-458c-b07e-56f1065d54cc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:50.862390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eea17d-5408-458c-b07e-56f1065d54cc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:50.862836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eea17d-5408-458c-b07e-56f1065d54cc None admin] PATH_INFO: `/v3/services/network` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:50.863645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eea17d-5408-458c-b07e-56f1065d54cc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:50.865430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eea17d-5408-458c-b07e-56f1065d54cc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e6eea17d-5408-458c-b07e-56f1065d54cc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:50.878176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:50.879373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 20:44:50.880299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:50.882849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 353/713] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:50 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:50.883910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eea17d-5408-458c-b07e-56f1065d54cc None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:50.914419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 361/714] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:50.915431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf613cd-8ffd-411d-a0d1-7abe8e6a9407 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:50.959820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:51.003367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:51.011783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eea17d-5408-458c-b07e-56f1065d54cc None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:51.025387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf613cd-8ffd-411d-a0d1-7abe8e6a9407 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:51.050548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72b261-583c-4460-86ee-b45d39a9a753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:51.051624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72b261-583c-4460-86ee-b45d39a9a753 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.052144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72b261-583c-4460-86ee-b45d39a9a753 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.052616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72b261-583c-4460-86ee-b45d39a9a753 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.063906 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72b261-583c-4460-86ee-b45d39a9a753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:51.065034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386fb83-5c4e-4d24-9e27-2ee2ea2a780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:51.065712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386fb83-5c4e-4d24-9e27-2ee2ea2a780e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.065992 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386fb83-5c4e-4d24-9e27-2ee2ea2a780e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.066261 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386fb83-5c4e-4d24-9e27-2ee2ea2a780e None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.066971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386fb83-5c4e-4d24-9e27-2ee2ea2a780e None admin] RBAC: Adding query filter params (name=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:51.067277 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386fb83-5c4e-4d24-9e27-2ee2ea2a780e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:51.068887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386fb83-5c4e-4d24-9e27-2ee2ea2a780e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-bb72b261-583c-4460-86ee-b45d39a9a753 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:51.073323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:51.074947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:51.075831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:51.078329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 362/716] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:44:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:51.078474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 354/716] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:51.079138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386fb83-5c4e-4d24-9e27-2ee2ea2a780e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:51.079640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72b261-583c-4460-86ee-b45d39a9a753 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:51.188462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72b261-583c-4460-86ee-b45d39a9a753 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:51.202732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386fb83-5c4e-4d24-9e27-2ee2ea2a780e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:51.212362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:51.229653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a7cd6-fc1f-40b1-bbb0-8fc99f1f5e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:51.230607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a7cd6-fc1f-40b1-bbb0-8fc99f1f5e59 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.231125 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a7cd6-fc1f-40b1-bbb0-8fc99f1f5e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.231615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a7cd6-fc1f-40b1-bbb0-8fc99f1f5e59 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.232633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a7cd6-fc1f-40b1-bbb0-8fc99f1f5e59 None admin] RBAC: Adding query filter params (type=network) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:51.233106 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a7cd6-fc1f-40b1-bbb0-8fc99f1f5e59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:51.235174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a7cd6-fc1f-40b1-bbb0-8fc99f1f5e59 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:51.242196 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 355/717] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:44:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:51.243283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a7cd6-fc1f-40b1-bbb0-8fc99f1f5e59 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:51.253821 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ed7c5-40b1-4031-853e-fa0c121467d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:51.254796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ed7c5-40b1-4031-853e-fa0c121467d2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.255120 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ed7c5-40b1-4031-853e-fa0c121467d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.255445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ed7c5-40b1-4031-853e-fa0c121467d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.269058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ed7c5-40b1-4031-853e-fa0c121467d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:51.272139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ed7c5-40b1-4031-853e-fa0c121467d2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:51.280284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f91ed7c5-40b1-4031-853e-fa0c121467d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:51.294137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:51.295088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:51.296022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:51.296022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:51.296022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:51.296022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:51.296782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 363/718] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:51.301499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ed7c5-40b1-4031-853e-fa0c121467d2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:51.481321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a7cd6-fc1f-40b1-bbb0-8fc99f1f5e59 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:51.513124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ed7c5-40b1-4031-853e-fa0c121467d2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:51.538920 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05bcaa-cc8a-429e-814a-41e80dfd6197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:51.539784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05bcaa-cc8a-429e-814a-41e80dfd6197 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.540071 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05bcaa-cc8a-429e-814a-41e80dfd6197 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.540381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05bcaa-cc8a-429e-814a-41e80dfd6197 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.541592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05bcaa-cc8a-429e-814a-41e80dfd6197 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:51.541909 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05bcaa-cc8a-429e-814a-41e80dfd6197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:51.543805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05bcaa-cc8a-429e-814a-41e80dfd6197 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:51.552763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526feabe-26f5-4319-a66a-9cf5ac3f83b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:51.553745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526feabe-26f5-4319-a66a-9cf5ac3f83b4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.554222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526feabe-26f5-4319-a66a-9cf5ac3f83b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.554691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526feabe-26f5-4319-a66a-9cf5ac3f83b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.555070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 356/719] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:51.560142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526feabe-26f5-4319-a66a-9cf5ac3f83b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:51.560258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ee446-0a1b-446e-8c94-8b20a0e90e1b None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.560258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ee446-0a1b-446e-8c94-8b20a0e90e1b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.560258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ee446-0a1b-446e-8c94-8b20a0e90e1b None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.560258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 357/720] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:51.560722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526feabe-26f5-4319-a66a-9cf5ac3f83b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:51.564823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ee446-0a1b-446e-8c94-8b20a0e90e1b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:51.564956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526feabe-26f5-4319-a66a-9cf5ac3f83b4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:51.580780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 364/721] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:51.587075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ce0fb-e473-4477-844b-863cc8035341 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.587743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ce0fb-e473-4477-844b-863cc8035341 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.588271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ce0fb-e473-4477-844b-863cc8035341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.606725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-261ce0fb-e473-4477-844b-863cc8035341 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:51.608495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-261ce0fb-e473-4477-844b-863cc8035341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:51.625102 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:51.670099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:51.692199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 365/722] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:51.693580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ce0fb-e473-4477-844b-863cc8035341 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:51.700034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ee446-0a1b-446e-8c94-8b20a0e90e1b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:51.738002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c869b-68f1-46b1-ad85-87d976fd409f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:51.738828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c869b-68f1-46b1-ad85-87d976fd409f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.739111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c869b-68f1-46b1-ad85-87d976fd409f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.739473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c869b-68f1-46b1-ad85-87d976fd409f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.751277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c869b-68f1-46b1-ad85-87d976fd409f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:51.752785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c869b-68f1-46b1-ad85-87d976fd409f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-5e3c869b-68f1-46b1-ad85-87d976fd409f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:51.771410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:51.772458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:51.773357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:51.773357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:51.773357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:51.773357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:51.776250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 358/723] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:51.777727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eced7f-b899-4a1f-976a-e5000e297ddf None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.778348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eced7f-b899-4a1f-976a-e5000e297ddf None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.778790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eced7f-b899-4a1f-976a-e5000e297ddf None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.785066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 359/724] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:51.785066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eced7f-b899-4a1f-976a-e5000e297ddf None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:51.818688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ce0fb-e473-4477-844b-863cc8035341 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:51.856014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b013e33-480f-4544-a64d-2c7b8eafa7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:51.858557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b013e33-480f-4544-a64d-2c7b8eafa7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.858846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b013e33-480f-4544-a64d-2c7b8eafa7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.859115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b013e33-480f-4544-a64d-2c7b8eafa7a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.866758 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b013e33-480f-4544-a64d-2c7b8eafa7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:51.868262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b013e33-480f-4544-a64d-2c7b8eafa7a8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-3b013e33-480f-4544-a64d-2c7b8eafa7a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:51.882908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:51.883849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:51.884787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:51.884787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:51.884787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:51.884787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:51.884787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:51.884787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:51.884787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:51.884787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:51.884787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 366/725] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:51.885921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b013e33-480f-4544-a64d-2c7b8eafa7a8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:51.907662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:51.914318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eced7f-b899-4a1f-976a-e5000e297ddf None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:51.939497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:51.955058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859568b9-e5b6-4348-8166-8c9d7c98a830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:51.955869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859568b9-e5b6-4348-8166-8c9d7c98a830 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.956172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859568b9-e5b6-4348-8166-8c9d7c98a830 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.956446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859568b9-e5b6-4348-8166-8c9d7c98a830 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:51.966998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859568b9-e5b6-4348-8166-8c9d7c98a830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-859568b9-e5b6-4348-8166-8c9d7c98a830 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:51.974897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:51.976002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:51.977058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:51.982704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 360/726] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:51.983999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd5a8-6aec-401f-b01f-a065e1987d48 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:51.984321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd5a8-6aec-401f-b01f-a065e1987d48 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:51.984601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd5a8-6aec-401f-b01f-a065e1987d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:52.000093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b013e33-480f-4544-a64d-2c7b8eafa7a8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:52.001322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-263dd5a8-6aec-401f-b01f-a065e1987d48 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:52.002502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-263dd5a8-6aec-401f-b01f-a065e1987d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:52.037549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2156bf-c6ff-419b-bfd2-63be343857d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:52.038886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2156bf-c6ff-419b-bfd2-63be343857d0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:52.038886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2156bf-c6ff-419b-bfd2-63be343857d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:52.038886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2156bf-c6ff-419b-bfd2-63be343857d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:52.039489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2156bf-c6ff-419b-bfd2-63be343857d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:52.039793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2156bf-c6ff-419b-bfd2-63be343857d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:52.041216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2156bf-c6ff-419b-bfd2-63be343857d0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:52.057693 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 367/727] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:52.057693 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2156bf-c6ff-419b-bfd2-63be343857d0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:52.077720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 361/728] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:52.078300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dd5a8-6aec-401f-b01f-a065e1987d48 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:52.161011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2156bf-c6ff-419b-bfd2-63be343857d0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:52.170397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:52.180678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:52.193966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dd5a8-6aec-401f-b01f-a065e1987d48 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:52.196970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e5245-7ae7-471d-8bd1-da2ff54b3097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:52.197640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e5245-7ae7-471d-8bd1-da2ff54b3097 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:52.197918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e5245-7ae7-471d-8bd1-da2ff54b3097 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:52.198186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e5245-7ae7-471d-8bd1-da2ff54b3097 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:52.199334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e5245-7ae7-471d-8bd1-da2ff54b3097 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:52.199637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e5245-7ae7-471d-8bd1-da2ff54b3097 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:52.201090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e5245-7ae7-471d-8bd1-da2ff54b3097 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:52.217666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 368/729] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:52.217666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e5245-7ae7-471d-8bd1-da2ff54b3097 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:52.235902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c888c99-703e-4a8f-8f46-5b5d54dac40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:52.236704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c888c99-703e-4a8f-8f46-5b5d54dac40e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:52.236983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c888c99-703e-4a8f-8f46-5b5d54dac40e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:52.237256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c888c99-703e-4a8f-8f46-5b5d54dac40e None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:52.238314 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c888c99-703e-4a8f-8f46-5b5d54dac40e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:52.238314 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c888c99-703e-4a8f-8f46-5b5d54dac40e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:52.239735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c888c99-703e-4a8f-8f46-5b5d54dac40e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:52.248590 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 362/730] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:52.249212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c888c99-703e-4a8f-8f46-5b5d54dac40e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:52.321971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e5245-7ae7-471d-8bd1-da2ff54b3097 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:52.353110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c888c99-703e-4a8f-8f46-5b5d54dac40e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:52.358438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894881f-fa9c-45f9-b11b-8600c1965ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:52.359276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894881f-fa9c-45f9-b11b-8600c1965ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:52.359559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894881f-fa9c-45f9-b11b-8600c1965ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:52.359837 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894881f-fa9c-45f9-b11b-8600c1965ea6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:52.367794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894881f-fa9c-45f9-b11b-8600c1965ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:52.369280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894881f-fa9c-45f9-b11b-8600c1965ea6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1894881f-fa9c-45f9-b11b-8600c1965ea6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:52.385284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:52.386229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:52.387138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:52.387138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:52.387138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:52.387138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:52.387138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:52.387138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:52.387138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:52.387138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:52.387138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:44:52.387138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:52.389158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 369/731] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jul 11 20:44:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:52.390057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894881f-fa9c-45f9-b11b-8600c1965ea6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:52.395747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd7e23-a2d4-4ae0-8c1d-497280944121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:52.396645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7e23-a2d4-4ae0-8c1d-497280944121 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:52.397797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7e23-a2d4-4ae0-8c1d-497280944121 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:52.398085 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7e23-a2d4-4ae0-8c1d-497280944121 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:52.398829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd7e23-a2d4-4ae0-8c1d-497280944121 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:52.400245 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd7e23-a2d4-4ae0-8c1d-497280944121 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:52.424154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 363/732] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:44:52 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:44:52.424154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd7e23-a2d4-4ae0-8c1d-497280944121 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:52.435284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:52.440794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:52.532205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894881f-fa9c-45f9-b11b-8600c1965ea6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:52.565781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd7e23-a2d4-4ae0-8c1d-497280944121 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:52.581161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e083432-1cbc-414f-808b-e03ac96cda40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:52.582253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e083432-1cbc-414f-808b-e03ac96cda40 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:52.582731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e083432-1cbc-414f-808b-e03ac96cda40 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:52.583242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e083432-1cbc-414f-808b-e03ac96cda40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:52.584555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e083432-1cbc-414f-808b-e03ac96cda40 None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=acb3ad858e4848b9a325c72c7328c863) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:52.585081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e083432-1cbc-414f-808b-e03ac96cda40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:52.586786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e083432-1cbc-414f-808b-e03ac96cda40 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:52.600225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 370/733] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:52 2022] GET /identity/v3/role_assignments?user.id=acb3ad858e4848b9a325c72c7328c863&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=60bd1c44ce154338a7face8949d07281 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:52.601561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e083432-1cbc-414f-808b-e03ac96cda40 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:52.610364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2052a6-ea12-42b7-8695-11a594dbeed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:52.611112 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2052a6-ea12-42b7-8695-11a594dbeed3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:52.611390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2052a6-ea12-42b7-8695-11a594dbeed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:52.611661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2052a6-ea12-42b7-8695-11a594dbeed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:52.628836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2052a6-ea12-42b7-8695-11a594dbeed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:52.631245 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2052a6-ea12-42b7-8695-11a594dbeed3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-7c2052a6-ea12-42b7-8695-11a594dbeed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:52.655464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:52.656476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:52.657348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:52.657348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:52.657348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:52.657348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:52.657348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 364/734] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:52.663681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2052a6-ea12-42b7-8695-11a594dbeed3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:52.720737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:52.732291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e083432-1cbc-414f-808b-e03ac96cda40 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:52.746910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:52.773378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc0a2d-db0c-4e36-b053-4b9b2926956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:52.774077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc0a2d-db0c-4e36-b053-4b9b2926956c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:52.775260 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc0a2d-db0c-4e36-b053-4b9b2926956c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:52.775539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc0a2d-db0c-4e36-b053-4b9b2926956c None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:52.776436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc0a2d-db0c-4e36-b053-4b9b2926956c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:52.776749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc0a2d-db0c-4e36-b053-4b9b2926956c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:52.778307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc0a2d-db0c-4e36-b053-4b9b2926956c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:52.799628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2052a6-ea12-42b7-8695-11a594dbeed3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:52.802779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 371/735] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jul 11 20:44:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:52.821128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc0a2d-db0c-4e36-b053-4b9b2926956c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:52.851452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e74f17-1736-473e-be38-823c1e8230df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:52.852914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e74f17-1736-473e-be38-823c1e8230df None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:52.853564 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e74f17-1736-473e-be38-823c1e8230df None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:52.854084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e74f17-1736-473e-be38-823c1e8230df None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:52.855239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e74f17-1736-473e-be38-823c1e8230df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:52.856077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e74f17-1736-473e-be38-823c1e8230df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:52.858197 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e74f17-1736-473e-be38-823c1e8230df None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:52.874431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 365/736] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:52.886763 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e74f17-1736-473e-be38-823c1e8230df None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:53.000618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e74f17-1736-473e-be38-823c1e8230df None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:53.018950 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc0a2d-db0c-4e36-b053-4b9b2926956c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:53.019899 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:53.039223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981992a-1a13-46ba-aadd-6b0753a03e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:53.040395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981992a-1a13-46ba-aadd-6b0753a03e77 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:53.040845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981992a-1a13-46ba-aadd-6b0753a03e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:53.041280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981992a-1a13-46ba-aadd-6b0753a03e77 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:53.049061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981992a-1a13-46ba-aadd-6b0753a03e77 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:53.050722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981992a-1a13-46ba-aadd-6b0753a03e77 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9981992a-1a13-46ba-aadd-6b0753a03e77 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:53.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:53.066716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:53.067699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:53.067699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:53.067699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:53.067699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:53.067699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:53.067699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:53.067699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:53.067699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:53.067699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:44:53.067699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:53.070351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 366/737] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:53.081387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981992a-1a13-46ba-aadd-6b0753a03e77 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:53.087543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193164c-8224-4da8-9b11-6b0b0a8a57db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:53.088238 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193164c-8224-4da8-9b11-6b0b0a8a57db None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:53.088519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193164c-8224-4da8-9b11-6b0b0a8a57db None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:53.088789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193164c-8224-4da8-9b11-6b0b0a8a57db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:53.109304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:53.218526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981992a-1a13-46ba-aadd-6b0753a03e77 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:53.225015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193164c-8224-4da8-9b11-6b0b0a8a57db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:53.227022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193164c-8224-4da8-9b11-6b0b0a8a57db None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:53.262804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780b014-a714-496b-945f-8165284132ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:53.264689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780b014-a714-496b-945f-8165284132ed None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:53.265177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780b014-a714-496b-945f-8165284132ed None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:53.265803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780b014-a714-496b-945f-8165284132ed None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:53.266864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780b014-a714-496b-945f-8165284132ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:53.267579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780b014-a714-496b-945f-8165284132ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:53.270194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780b014-a714-496b-945f-8165284132ed None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:53.310237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 367/738] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jul 11 20:44:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:53.317800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780b014-a714-496b-945f-8165284132ed None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:53.384892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:53.430218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 372/739] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:52 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 20:44:53.435362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193164c-8224-4da8-9b11-6b0b0a8a57db None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:53.476068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780b014-a714-496b-945f-8165284132ed None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:53.497665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:53.511735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee35fc-1b87-4450-adf3-a17837279f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:53.512444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee35fc-1b87-4450-adf3-a17837279f9f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:53.512748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee35fc-1b87-4450-adf3-a17837279f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:53.512988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee35fc-1b87-4450-adf3-a17837279f9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:53.555466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193164c-8224-4da8-9b11-6b0b0a8a57db None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:53.585873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee35fc-1b87-4450-adf3-a17837279f9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:53.587363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee35fc-1b87-4450-adf3-a17837279f9f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:53.594352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0205e-b262-4ae4-ad46-c27dc94cc28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:53.595512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0205e-b262-4ae4-ad46-c27dc94cc28f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:53.596044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0205e-b262-4ae4-ad46-c27dc94cc28f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:53.596571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0205e-b262-4ae4-ad46-c27dc94cc28f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:53.608127 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0205e-b262-4ae4-ad46-c27dc94cc28f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:53.609957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0205e-b262-4ae4-ad46-c27dc94cc28f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:53.623447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6fc0205e-b262-4ae4-ad46-c27dc94cc28f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:53.631332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:53.632306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:53.633265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:53.633265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:53.633265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:53.633265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:53.635662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 373/740] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:53.637379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9da2-e38f-4242-8136-d85c221d72ee None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:53.637870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9da2-e38f-4242-8136-d85c221d72ee None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:53.638335 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9da2-e38f-4242-8136-d85c221d72ee None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:53.644345 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 374/741] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:53.645357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9da2-e38f-4242-8136-d85c221d72ee None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:53.694457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 368/742] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:44:53 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 20:44:53.695916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff83765-6213-49d4-a089-f424f1aaddef None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:53.696204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff83765-6213-49d4-a089-f424f1aaddef None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:53.696485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff83765-6213-49d4-a089-f424f1aaddef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:53.714409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5ff83765-6213-49d4-a089-f424f1aaddef None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:53.715601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5ff83765-6213-49d4-a089-f424f1aaddef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:53.752069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9da2-e38f-4242-8136-d85c221d72ee None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:53.754933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:53.788817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec6c3e-cc0d-48e1-b69b-e69b1df86962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:53.789637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec6c3e-cc0d-48e1-b69b-e69b1df86962 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:53.790017 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec6c3e-cc0d-48e1-b69b-e69b1df86962 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:53.790509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec6c3e-cc0d-48e1-b69b-e69b1df86962 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:53.790876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 369/743] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 20:44:53.791654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec6c3e-cc0d-48e1-b69b-e69b1df86962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:53.792336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff83765-6213-49d4-a089-f424f1aaddef None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:53.792904 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec6c3e-cc0d-48e1-b69b-e69b1df86962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:53.801527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec6c3e-cc0d-48e1-b69b-e69b1df86962 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:53.820919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 375/744] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:53.825733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec6c3e-cc0d-48e1-b69b-e69b1df86962 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:53.874905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:53.909349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff83765-6213-49d4-a089-f424f1aaddef None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:53.936027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec6c3e-cc0d-48e1-b69b-e69b1df86962 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:53.946938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa3f1f-ef0e-4d90-bfe7-faec9c25e98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:53.948896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa3f1f-ef0e-4d90-bfe7-faec9c25e98b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:53.949356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa3f1f-ef0e-4d90-bfe7-faec9c25e98b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:53.949803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa3f1f-ef0e-4d90-bfe7-faec9c25e98b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:53.961771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa3f1f-ef0e-4d90-bfe7-faec9c25e98b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:53.963524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa3f1f-ef0e-4d90-bfe7-faec9c25e98b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:53.972884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6180a-dce4-44ab-8050-d7d8163fa6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:53.973977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6180a-dce4-44ab-8050-d7d8163fa6fc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:53.974537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6180a-dce4-44ab-8050-d7d8163fa6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:53.975037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6180a-dce4-44ab-8050-d7d8163fa6fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:53.983080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6180a-dce4-44ab-8050-d7d8163fa6fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:53.984767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6180a-dce4-44ab-8050-d7d8163fa6fc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f9fa3f1f-ef0e-4d90-bfe7-faec9c25e98b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:53.985950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:53.986809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:53.987729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:53.987729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:53.987729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:53.987729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:53.990171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 370/745] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:53.991222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa3f1f-ef0e-4d90-bfe7-faec9c25e98b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-08f6180a-dce4-44ab-8050-d7d8163fa6fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:54.002519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:54.003408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:54.004435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:54.004435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:54.004435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:54.004435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:54.004435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:54.004435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:54.004435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:54.004435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:54.004435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 376/746] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:54.005059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6180a-dce4-44ab-8050-d7d8163fa6fc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:54.030333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:54.101406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa3f1f-ef0e-4d90-bfe7-faec9c25e98b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:54.106432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6180a-dce4-44ab-8050-d7d8163fa6fc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:54.132502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:54.140684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ea46e-d812-4b2f-a337-7f2827931f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:54.141572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ea46e-d812-4b2f-a337-7f2827931f5d None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:54.141886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ea46e-d812-4b2f-a337-7f2827931f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:54.142120 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ea46e-d812-4b2f-a337-7f2827931f5d None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:54.142905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ea46e-d812-4b2f-a337-7f2827931f5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:54.144810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b0480-f4ba-443b-ba0b-34650d2ff49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:54.144933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ea46e-d812-4b2f-a337-7f2827931f5d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:54.145890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b0480-f4ba-443b-ba0b-34650d2ff49d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:54.146349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b0480-f4ba-443b-ba0b-34650d2ff49d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:54.146797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b0480-f4ba-443b-ba0b-34650d2ff49d None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:54.147955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b0480-f4ba-443b-ba0b-34650d2ff49d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:54.148559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b0480-f4ba-443b-ba0b-34650d2ff49d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:54.150198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b0480-f4ba-443b-ba0b-34650d2ff49d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:54.163342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 371/747] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:44:53 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:44:54.168613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ea46e-d812-4b2f-a337-7f2827931f5d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:54.168782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 377/748] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:54.171018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b0480-f4ba-443b-ba0b-34650d2ff49d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:54.291212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ea46e-d812-4b2f-a337-7f2827931f5d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:54.333725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cdef5-2d4d-4fbb-9002-503beb8bcea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:54.334465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cdef5-2d4d-4fbb-9002-503beb8bcea8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:54.334757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cdef5-2d4d-4fbb-9002-503beb8bcea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:54.335806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cdef5-2d4d-4fbb-9002-503beb8bcea8 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:54.336656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cdef5-2d4d-4fbb-9002-503beb8bcea8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:54.336968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cdef5-2d4d-4fbb-9002-503beb8bcea8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:54.339643 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cdef5-2d4d-4fbb-9002-503beb8bcea8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:54.368447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 372/749] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:54.372992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cdef5-2d4d-4fbb-9002-503beb8bcea8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:54.377680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:54.412745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b0480-f4ba-443b-ba0b-34650d2ff49d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:54.423290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:54.462786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975e9a9-157e-4bc3-bb54-c96c92a0942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:54.464017 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975e9a9-157e-4bc3-bb54-c96c92a0942a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:54.464330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975e9a9-157e-4bc3-bb54-c96c92a0942a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:54.464852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975e9a9-157e-4bc3-bb54-c96c92a0942a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:54.474124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975e9a9-157e-4bc3-bb54-c96c92a0942a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:54.477460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975e9a9-157e-4bc3-bb54-c96c92a0942a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c975e9a9-157e-4bc3-bb54-c96c92a0942a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:54.496200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:54.497121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:54.497983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:54.497983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:54.497983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:54.497983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:54.497983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:54.497983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:54.497983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:54.497983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:54.498400 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 378/750] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:54.506113 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975e9a9-157e-4bc3-bb54-c96c92a0942a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:54.512382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cdef5-2d4d-4fbb-9002-503beb8bcea8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:54.558475 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c60ed-92b1-4f59-8975-a38cef2e0dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:54.559077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c60ed-92b1-4f59-8975-a38cef2e0dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:54.559353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c60ed-92b1-4f59-8975-a38cef2e0dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:54.559621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c60ed-92b1-4f59-8975-a38cef2e0dc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:54.560506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c60ed-92b1-4f59-8975-a38cef2e0dc7 None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=812f2c4a42fd470c9c57bd14310f320f) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:54.560878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c60ed-92b1-4f59-8975-a38cef2e0dc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:54.562382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c60ed-92b1-4f59-8975-a38cef2e0dc7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:54.569796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 373/751] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:54 2022] GET /identity/v3/role_assignments?user.id=812f2c4a42fd470c9c57bd14310f320f&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:54.636183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975e9a9-157e-4bc3-bb54-c96c92a0942a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:54.674876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b533e1-5774-4f9d-8f70-b7228fc11e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:54.675814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b533e1-5774-4f9d-8f70-b7228fc11e2a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:54.676328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b533e1-5774-4f9d-8f70-b7228fc11e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:54.676822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b533e1-5774-4f9d-8f70-b7228fc11e2a None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:54.678180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b533e1-5774-4f9d-8f70-b7228fc11e2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:54.678667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b533e1-5774-4f9d-8f70-b7228fc11e2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:54.681002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b533e1-5774-4f9d-8f70-b7228fc11e2a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:54.702287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 379/752] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:54.710180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c60ed-92b1-4f59-8975-a38cef2e0dc7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:54.820505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:54.825169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c60ed-92b1-4f59-8975-a38cef2e0dc7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:54.862189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9fd11-252e-4a07-99be-1e0ea8d05f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:54.862869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9fd11-252e-4a07-99be-1e0ea8d05f8d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:54.863191 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9fd11-252e-4a07-99be-1e0ea8d05f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:54.863465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9fd11-252e-4a07-99be-1e0ea8d05f8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:54.864470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9fd11-252e-4a07-99be-1e0ea8d05f8d None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=97b5f28d84fe4570a7c5c904dc4e502c) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:54.864786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9fd11-252e-4a07-99be-1e0ea8d05f8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:54.866251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9fd11-252e-4a07-99be-1e0ea8d05f8d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:54.873063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 374/753] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:44:54 2022] GET /identity/v3/role_assignments?user.id=97b5f28d84fe4570a7c5c904dc4e502c&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=60bd1c44ce154338a7face8949d07281 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:54.892429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd0149-d6f8-4bca-b4d0-1116b909c1c1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:54.892751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd0149-d6f8-4bca-b4d0-1116b909c1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:54.892981 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd0149-d6f8-4bca-b4d0-1116b909c1c1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:54.894740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 380/754] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:54.900742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e11f3-606e-40ce-be61-a45e160c1c9d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:54.901296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e11f3-606e-40ce-be61-a45e160c1c9d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:54.901728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e11f3-606e-40ce-be61-a45e160c1c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:54.921139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ba4e11f3-606e-40ce-be61-a45e160c1c9d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:54.922586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ba4e11f3-606e-40ce-be61-a45e160c1c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:54.998286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 375/755] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:44:55.021228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd0149-d6f8-4bca-b4d0-1116b909c1c1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:55.024484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:55.074503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a1dc-6bd0-45d5-a9dd-78dc547754af None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:55.075122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a1dc-6bd0-45d5-a9dd-78dc547754af None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:55.075590 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a1dc-6bd0-45d5-a9dd-78dc547754af None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:55.077465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 376/756] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:55.082637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c72e6-4303-4f0b-8cd5-b47d1fa35ef9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:55.083083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c72e6-4303-4f0b-8cd5-b47d1fa35ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:55.083368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c72e6-4303-4f0b-8cd5-b47d1fa35ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:55.104710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-891c72e6-4303-4f0b-8cd5-b47d1fa35ef9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:55.105892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-891c72e6-4303-4f0b-8cd5-b47d1fa35ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:55.150498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd0149-d6f8-4bca-b4d0-1116b909c1c1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:55.194097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56c0f2-a30c-4aa8-b14a-642625a8d984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:55.194908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56c0f2-a30c-4aa8-b14a-642625a8d984 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:55.195235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56c0f2-a30c-4aa8-b14a-642625a8d984 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:55.195512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56c0f2-a30c-4aa8-b14a-642625a8d984 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:55.196214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 377/757] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:44:55.206911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56c0f2-a30c-4aa8-b14a-642625a8d984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:55.214473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c72e6-4303-4f0b-8cd5-b47d1fa35ef9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7e56c0f2-a30c-4aa8-b14a-642625a8d984 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:55.215014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:55.215878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:55.216748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 381/758] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:55.225117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56c0f2-a30c-4aa8-b14a-642625a8d984 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:55.254971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:55.350096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c72e6-4303-4f0b-8cd5-b47d1fa35ef9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:55.361853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:55.387760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59cdf0-a9fb-4214-900a-91d75433fe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:55.388693 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59cdf0-a9fb-4214-900a-91d75433fe36 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:55.388979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59cdf0-a9fb-4214-900a-91d75433fe36 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:55.389264 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59cdf0-a9fb-4214-900a-91d75433fe36 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:55.389890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59cdf0-a9fb-4214-900a-91d75433fe36 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:55.391259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59cdf0-a9fb-4214-900a-91d75433fe36 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9d59cdf0-a9fb-4214-900a-91d75433fe36 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:55.403916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:55.406061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 20:44:55.407984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:55.416453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 378/759] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Jul 11 20:44:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:55.417415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59cdf0-a9fb-4214-900a-91d75433fe36 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:55.437908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56c0f2-a30c-4aa8-b14a-642625a8d984 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:55.501270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a7dd-85a3-4402-98e9-eef41f376363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:55.502015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a7dd-85a3-4402-98e9-eef41f376363 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:55.502302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a7dd-85a3-4402-98e9-eef41f376363 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:55.502586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a7dd-85a3-4402-98e9-eef41f376363 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:55.503405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38a7dd-85a3-4402-98e9-eef41f376363 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:55.503717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38a7dd-85a3-4402-98e9-eef41f376363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:55.505869 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38a7dd-85a3-4402-98e9-eef41f376363 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:55.524460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 382/760] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:55.533450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a7dd-85a3-4402-98e9-eef41f376363 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:55.553790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:55.570446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59cdf0-a9fb-4214-900a-91d75433fe36 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:55.605933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa46c9b-3772-4c7c-abf5-539d7d292581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:55.606626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa46c9b-3772-4c7c-abf5-539d7d292581 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:55.606900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa46c9b-3772-4c7c-abf5-539d7d292581 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:55.607169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa46c9b-3772-4c7c-abf5-539d7d292581 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:55.607881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa46c9b-3772-4c7c-abf5-539d7d292581 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:55.608207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa46c9b-3772-4c7c-abf5-539d7d292581 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:55.609583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa46c9b-3772-4c7c-abf5-539d7d292581 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:55.615181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 379/761] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jul 11 20:44:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:55.620677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa46c9b-3772-4c7c-abf5-539d7d292581 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:55.698714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a7dd-85a3-4402-98e9-eef41f376363 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:55.748171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:55.769513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a299-5266-4300-942b-7077acf279f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:55.770229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a299-5266-4300-942b-7077acf279f5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:55.770671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a299-5266-4300-942b-7077acf279f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:55.771033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a299-5266-4300-942b-7077acf279f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:55.800872 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236a299-5266-4300-942b-7077acf279f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:55.802528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236a299-5266-4300-942b-7077acf279f5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:55.810296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa46c9b-3772-4c7c-abf5-539d7d292581 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d236a299-5266-4300-942b-7077acf279f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:55.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:55.841919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:55.842785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:55.842785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:55.842785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:55.842785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:55.845002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 383/762] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:55.849538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f09b84-7e6b-4b46-bbcd-65093d0b588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:55.849538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f09b84-7e6b-4b46-bbcd-65093d0b588a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:55.849538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f09b84-7e6b-4b46-bbcd-65093d0b588a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:55.849538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f09b84-7e6b-4b46-bbcd-65093d0b588a None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:55.850262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f09b84-7e6b-4b46-bbcd-65093d0b588a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:55.850578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f09b84-7e6b-4b46-bbcd-65093d0b588a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:55.851312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a299-5266-4300-942b-7077acf279f5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:55.852023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f09b84-7e6b-4b46-bbcd-65093d0b588a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:55.857883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 380/763] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jul 11 20:44:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:55.869158 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f09b84-7e6b-4b46-bbcd-65093d0b588a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:55.892736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:55.952873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a299-5266-4300-942b-7077acf279f5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:55.986598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a1535-510f-4a44-b455-0fdc91790c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:55.987600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a1535-510f-4a44-b455-0fdc91790c22 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:55.988071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a1535-510f-4a44-b455-0fdc91790c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:55.988572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a1535-510f-4a44-b455-0fdc91790c22 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:55.989526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a1535-510f-4a44-b455-0fdc91790c22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:55.990020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a1535-510f-4a44-b455-0fdc91790c22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:55.991594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a1535-510f-4a44-b455-0fdc91790c22 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:56.004021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 384/764] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:56.009795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a1535-510f-4a44-b455-0fdc91790c22 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:56.044496 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:56.057052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f09b84-7e6b-4b46-bbcd-65093d0b588a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:56.112450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a1535-510f-4a44-b455-0fdc91790c22 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:56.147302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27c8cc-c644-4d87-b40d-3534735b6c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:56.148557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27c8cc-c644-4d87-b40d-3534735b6c42 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:56.149048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27c8cc-c644-4d87-b40d-3534735b6c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:56.149505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27c8cc-c644-4d87-b40d-3534735b6c42 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:56.157062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27c8cc-c644-4d87-b40d-3534735b6c42 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:56.159144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27c8cc-c644-4d87-b40d-3534735b6c42 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:56.160505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4c61c-5d0b-4be5-a6b9-dd365bb90f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:56.161348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4c61c-5d0b-4be5-a6b9-dd365bb90f03 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:56.161543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4c61c-5d0b-4be5-a6b9-dd365bb90f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:56.161814 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4c61c-5d0b-4be5-a6b9-dd365bb90f03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:56.163063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4c61c-5d0b-4be5-a6b9-dd365bb90f03 None admin] RBAC: Adding query filter params (interface=public, service_id=0443d7a49ccb43f2bff341db7d4393a7, region_id=RegionOne) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:56.163063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4c61c-5d0b-4be5-a6b9-dd365bb90f03 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:56.164330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4c61c-5d0b-4be5-a6b9-dd365bb90f03 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6e27c8cc-c644-4d87-b40d-3534735b6c42 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:56.175573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:56.176968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:56.177914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:56.177914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:56.177914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:56.177914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:56.177914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:56.177914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:56.177914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:56.177914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:56.177914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:44:56.177914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:56.179379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 381/765] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jul 11 20:44:55 2022] GET /identity/v3/endpoints?service_id=0443d7a49ccb43f2bff341db7d4393a7&interface=public®ion_id=RegionOne => generated 194 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:56.180684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 385/766] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jul 11 20:44:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:56.186273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4c61c-5d0b-4be5-a6b9-dd365bb90f03 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:56.190208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27c8cc-c644-4d87-b40d-3534735b6c42 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:56.295676 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:56.317960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4c61c-5d0b-4be5-a6b9-dd365bb90f03 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:56.352628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989df15d-d62f-45b4-9787-6f5fa8f3310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:56.353306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989df15d-d62f-45b4-9787-6f5fa8f3310c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:56.353588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989df15d-d62f-45b4-9787-6f5fa8f3310c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:56.353860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989df15d-d62f-45b4-9787-6f5fa8f3310c None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:56.354558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989df15d-d62f-45b4-9787-6f5fa8f3310c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:56.355941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989df15d-d62f-45b4-9787-6f5fa8f3310c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:56.372835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 382/767] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jul 11 20:44:56 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:56.379544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27c8cc-c644-4d87-b40d-3534735b6c42 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:56.388535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:56.417636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2a144-a16b-4e1d-8235-28bb115a7664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:56.418349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2a144-a16b-4e1d-8235-28bb115a7664 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:56.418632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2a144-a16b-4e1d-8235-28bb115a7664 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:56.418908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2a144-a16b-4e1d-8235-28bb115a7664 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:56.419777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2a144-a16b-4e1d-8235-28bb115a7664 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:56.420088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2a144-a16b-4e1d-8235-28bb115a7664 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:56.421591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2a144-a16b-4e1d-8235-28bb115a7664 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:56.441416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 386/768] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jul 11 20:44:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:56.449658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989df15d-d62f-45b4-9787-6f5fa8f3310c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:56.599314 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989df15d-d62f-45b4-9787-6f5fa8f3310c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:56.639439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaae7e4-e0e4-4523-a903-95f971a76192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:56.640251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaae7e4-e0e4-4523-a903-95f971a76192 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:56.640534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaae7e4-e0e4-4523-a903-95f971a76192 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:56.640806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaae7e4-e0e4-4523-a903-95f971a76192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:56.652905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaae7e4-e0e4-4523-a903-95f971a76192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:56.654383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaae7e4-e0e4-4523-a903-95f971a76192 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:56.657778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-2aaae7e4-e0e4-4523-a903-95f971a76192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:56.674828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:56.675779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:56.677792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:56.677792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:56.677792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:56.677792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:56.677792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 383/769] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:56.682507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2a144-a16b-4e1d-8235-28bb115a7664 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:56.792218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2a144-a16b-4e1d-8235-28bb115a7664 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:56.838447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045a2b1-e570-4a90-aa86-c81a866de6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:56.839140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045a2b1-e570-4a90-aa86-c81a866de6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:56.839424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045a2b1-e570-4a90-aa86-c81a866de6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:56.839697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045a2b1-e570-4a90-aa86-c81a866de6e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:56.840472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045a2b1-e570-4a90-aa86-c81a866de6e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:56.840779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045a2b1-e570-4a90-aa86-c81a866de6e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:56.842174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045a2b1-e570-4a90-aa86-c81a866de6e6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:56.858831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 387/770] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:56.865249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaae7e4-e0e4-4523-a903-95f971a76192 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:56.976015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaae7e4-e0e4-4523-a903-95f971a76192 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:57.010481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c0db9-dcda-43c6-aa77-7491da5f4207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:57.011307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c0db9-dcda-43c6-aa77-7491da5f4207 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.011638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c0db9-dcda-43c6-aa77-7491da5f4207 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.011912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c0db9-dcda-43c6-aa77-7491da5f4207 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.019045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c0db9-dcda-43c6-aa77-7491da5f4207 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:57.020566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c0db9-dcda-43c6-aa77-7491da5f4207 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d57c0db9-dcda-43c6-aa77-7491da5f4207 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:57.035405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:57.036656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:57.037723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:57.037723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:57.037723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:57.037723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:57.037723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:57.037723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:57.037723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:57.037723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:57.037723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 384/771] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:57.042742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd434578-2a35-411c-b466-ea78699912c5 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.042969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c0db9-dcda-43c6-aa77-7491da5f4207 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:57.043125 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd434578-2a35-411c-b466-ea78699912c5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.043297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd434578-2a35-411c-b466-ea78699912c5 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.048723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 388/772] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:57.050278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4cb3d-d979-4e0f-a10c-7ba6d3af3187 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.050565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4cb3d-d979-4e0f-a10c-7ba6d3af3187 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.050848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4cb3d-d979-4e0f-a10c-7ba6d3af3187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.068092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-01a4cb3d-d979-4e0f-a10c-7ba6d3af3187 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:57.069549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-01a4cb3d-d979-4e0f-a10c-7ba6d3af3187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:57.092394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:57.100996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:57.146293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c0db9-dcda-43c6-aa77-7491da5f4207 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:57.182219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660b7c3-60cb-4040-bbc9-d002dc11f197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:57.182894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660b7c3-60cb-4040-bbc9-d002dc11f197 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.183177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660b7c3-60cb-4040-bbc9-d002dc11f197 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.183450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660b7c3-60cb-4040-bbc9-d002dc11f197 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.184297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660b7c3-60cb-4040-bbc9-d002dc11f197 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:57.184601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660b7c3-60cb-4040-bbc9-d002dc11f197 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:57.186119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660b7c3-60cb-4040-bbc9-d002dc11f197 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:57.201383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 385/773] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:44:57.204883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d90340-fb18-4907-bf66-8fdeaa1c19d1 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.205173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d90340-fb18-4907-bf66-8fdeaa1c19d1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.205453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d90340-fb18-4907-bf66-8fdeaa1c19d1 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.208232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 386/774] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:57.209033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d90340-fb18-4907-bf66-8fdeaa1c19d1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:57.228301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 389/775] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:44:57.229672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799a2e0-d5a4-4e8c-a327-ed7cf273b148 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.229971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799a2e0-d5a4-4e8c-a327-ed7cf273b148 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.230248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799a2e0-d5a4-4e8c-a327-ed7cf273b148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.262597 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2799a2e0-d5a4-4e8c-a327-ed7cf273b148 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:57.263783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2799a2e0-d5a4-4e8c-a327-ed7cf273b148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:57.322872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d90340-fb18-4907-bf66-8fdeaa1c19d1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:57.335998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 390/776] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:44:57.341026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799a2e0-d5a4-4e8c-a327-ed7cf273b148 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:57.355210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:57.365376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792787a3-8c36-4c31-88e5-ab97d5ed06dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:57.366291 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792787a3-8c36-4c31-88e5-ab97d5ed06dd None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.366574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792787a3-8c36-4c31-88e5-ab97d5ed06dd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.366844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792787a3-8c36-4c31-88e5-ab97d5ed06dd None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/acb3ad858e4848b9a325c72c7328c863/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.369043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792787a3-8c36-4c31-88e5-ab97d5ed06dd None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=acb3ad858e4848b9a325c72c7328c863, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:57.370509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792787a3-8c36-4c31-88e5-ab97d5ed06dd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:57.385826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 387/777] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:44:57 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/acb3ad858e4848b9a325c72c7328c863/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:44:57.387117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5ae0d-6541-4add-bbc4-3c452143433b None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.387401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5ae0d-6541-4add-bbc4-3c452143433b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.387673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5ae0d-6541-4add-bbc4-3c452143433b None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.389114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 388/778] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:57.393998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5ae0d-6541-4add-bbc4-3c452143433b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:57.440611 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:57.471034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799a2e0-d5a4-4e8c-a327-ed7cf273b148 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:57.518879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0191bf-b708-4275-8a8f-95a40c8f4dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:57.519874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0191bf-b708-4275-8a8f-95a40c8f4dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.520201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0191bf-b708-4275-8a8f-95a40c8f4dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.520882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0191bf-b708-4275-8a8f-95a40c8f4dd1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.521818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0191bf-b708-4275-8a8f-95a40c8f4dd1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:57.523492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0191bf-b708-4275-8a8f-95a40c8f4dd1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:57.525393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5ae0d-6541-4add-bbc4-3c452143433b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-cd0191bf-b708-4275-8a8f-95a40c8f4dd1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:57.541849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:57.542892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:57.543729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 391/779] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:57 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:57.545867 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd1817-8692-485b-8988-a05fab59af0f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.546411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd1817-8692-485b-8988-a05fab59af0f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.546937 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd1817-8692-485b-8988-a05fab59af0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.570273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-23dd1817-8692-485b-8988-a05fab59af0f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:57.571846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-23dd1817-8692-485b-8988-a05fab59af0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:57.573594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ffec9-9807-4b40-b541-da3d1e615c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:57.574707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ffec9-9807-4b40-b541-da3d1e615c9f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.574999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ffec9-9807-4b40-b541-da3d1e615c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.575276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ffec9-9807-4b40-b541-da3d1e615c9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.600227 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ffec9-9807-4b40-b541-da3d1e615c9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-783ffec9-9807-4b40-b541-da3d1e615c9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:57.610501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:57.611717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:44:57.612784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:57.615572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 389/780] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:44:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:57.615572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ffec9-9807-4b40-b541-da3d1e615c9f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:57.653804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:57.670449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 392/781] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:44:57.671737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd1817-8692-485b-8988-a05fab59af0f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:57.745470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ffec9-9807-4b40-b541-da3d1e615c9f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:57.748972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:57.782783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e452f-3846-448a-ae78-0a60252b6e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:57.783388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e452f-3846-448a-ae78-0a60252b6e0e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.783667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e452f-3846-448a-ae78-0a60252b6e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.784208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e452f-3846-448a-ae78-0a60252b6e0e None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.784694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e452f-3846-448a-ae78-0a60252b6e0e None admin] RBAC: Adding query filter params (name=network) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:57.785003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e452f-3846-448a-ae78-0a60252b6e0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:57.786789 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e452f-3846-448a-ae78-0a60252b6e0e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:57.788475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd1817-8692-485b-8988-a05fab59af0f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:57.792034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 390/782] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:44:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:57.792878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e452f-3846-448a-ae78-0a60252b6e0e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:57.827526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a7e32-078c-430b-b33a-3ccbd4ab1705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:57.828653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a7e32-078c-430b-b33a-3ccbd4ab1705 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.829162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a7e32-078c-430b-b33a-3ccbd4ab1705 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.829873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a7e32-078c-430b-b33a-3ccbd4ab1705 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.830970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a7e32-078c-430b-b33a-3ccbd4ab1705 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:57.831581 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a7e32-078c-430b-b33a-3ccbd4ab1705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:57.834014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a7e32-078c-430b-b33a-3ccbd4ab1705 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:57.843217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 393/783] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:44:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:57.844061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a7e32-078c-430b-b33a-3ccbd4ab1705 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:57.907626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:57.913187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e452f-3846-448a-ae78-0a60252b6e0e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:57.955392 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5dd69-e782-4d36-b398-5799d48bfdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:57.956677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5dd69-e782-4d36-b398-5799d48bfdec None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.957108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5dd69-e782-4d36-b398-5799d48bfdec None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.957514 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5dd69-e782-4d36-b398-5799d48bfdec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.959862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a7e32-078c-430b-b33a-3ccbd4ab1705 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:57.969245 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5dd69-e782-4d36-b398-5799d48bfdec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-6ab5dd69-e782-4d36-b398-5799d48bfdec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:44:57.977775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:57.978860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:44:57.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:57.982231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 391/784] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:57.983379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5dd69-e782-4d36-b398-5799d48bfdec None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:57.989374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:57.995185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19524dfc-ddff-43fc-a047-8a8fbb23c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:57.995649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19524dfc-ddff-43fc-a047-8a8fbb23c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:57.995932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19524dfc-ddff-43fc-a047-8a8fbb23c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:57.996373 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19524dfc-ddff-43fc-a047-8a8fbb23c8c6 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:57.997100 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19524dfc-ddff-43fc-a047-8a8fbb23c8c6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:57.997408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19524dfc-ddff-43fc-a047-8a8fbb23c8c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:57.998989 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19524dfc-ddff-43fc-a047-8a8fbb23c8c6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.009646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 394/785] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:44:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:58.010528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19524dfc-ddff-43fc-a047-8a8fbb23c8c6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:58.098697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5dd69-e782-4d36-b398-5799d48bfdec None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:58.111340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19524dfc-ddff-43fc-a047-8a8fbb23c8c6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:58.136571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638d27-fc0d-4f24-a35f-f6cf1e361e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:58.137800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638d27-fc0d-4f24-a35f-f6cf1e361e8b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.138347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638d27-fc0d-4f24-a35f-f6cf1e361e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.138806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638d27-fc0d-4f24-a35f-f6cf1e361e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.146004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3adf1-73df-4f79-b57c-09f48ef57748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:58.146761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3adf1-73df-4f79-b57c-09f48ef57748 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.147401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3adf1-73df-4f79-b57c-09f48ef57748 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.147401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3adf1-73df-4f79-b57c-09f48ef57748 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.148058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3adf1-73df-4f79-b57c-09f48ef57748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:58.148390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3adf1-73df-4f79-b57c-09f48ef57748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:58.149817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3adf1-73df-4f79-b57c-09f48ef57748 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.152073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86638d27-fc0d-4f24-a35f-f6cf1e361e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:58.154937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86638d27-fc0d-4f24-a35f-f6cf1e361e8b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.158900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 395/786] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:44:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:58.159812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3adf1-73df-4f79-b57c-09f48ef57748 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-86638d27-fc0d-4f24-a35f-f6cf1e361e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:58.174872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:58.175772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:58.176840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:58.176840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:58.176840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:58.176840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:58.179151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 392/787] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:58.180169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638d27-fc0d-4f24-a35f-f6cf1e361e8b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:58.197209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:58.222291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:58.267058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3adf1-73df-4f79-b57c-09f48ef57748 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:58.298996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638d27-fc0d-4f24-a35f-f6cf1e361e8b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:58.301552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf4f46-9b52-4051-b708-8161e3ef387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:58.302639 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4f46-9b52-4051-b708-8161e3ef387c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.302750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4f46-9b52-4051-b708-8161e3ef387c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.302941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4f46-9b52-4051-b708-8161e3ef387c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.303745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf4f46-9b52-4051-b708-8161e3ef387c None admin] RBAC: Adding query filter params (interface=public, service_id=4085f7dcf2634258943d40863caf0192, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:58.304059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf4f46-9b52-4051-b708-8161e3ef387c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:58.305463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf4f46-9b52-4051-b708-8161e3ef387c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.318893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 396/788] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jul 11 20:44:58 2022] GET /identity/v3/endpoints?service_id=4085f7dcf2634258943d40863caf0192&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:58.319995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf4f46-9b52-4051-b708-8161e3ef387c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:58.337918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fb4c1-8711-4ff3-a35b-9eed6cd3918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:58.339044 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fb4c1-8711-4ff3-a35b-9eed6cd3918f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.339542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fb4c1-8711-4ff3-a35b-9eed6cd3918f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.340012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fb4c1-8711-4ff3-a35b-9eed6cd3918f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.352455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760fb4c1-8711-4ff3-a35b-9eed6cd3918f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:58.354391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760fb4c1-8711-4ff3-a35b-9eed6cd3918f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-760fb4c1-8711-4ff3-a35b-9eed6cd3918f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:58.373067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:58.373984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:58.374879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:58.374879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:58.374879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:58.374879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:58.377178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 393/789] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:58.378351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fb4c1-8711-4ff3-a35b-9eed6cd3918f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:58.423702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf4f46-9b52-4051-b708-8161e3ef387c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:58.444201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:58.447441 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:58.459336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628580d-5f31-4324-b771-4c9f43ebecea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:58.460137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628580d-5f31-4324-b771-4c9f43ebecea None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.460316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628580d-5f31-4324-b771-4c9f43ebecea None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.460831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628580d-5f31-4324-b771-4c9f43ebecea None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.461471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628580d-5f31-4324-b771-4c9f43ebecea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:58.461745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628580d-5f31-4324-b771-4c9f43ebecea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:58.463139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628580d-5f31-4324-b771-4c9f43ebecea None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.480460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 397/790] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:58.481373 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628580d-5f31-4324-b771-4c9f43ebecea None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:58.494973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fb4c1-8711-4ff3-a35b-9eed6cd3918f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:58.541822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ecb90-3f30-47ef-8b85-3cb72c449250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:58.542496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ecb90-3f30-47ef-8b85-3cb72c449250 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.542771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ecb90-3f30-47ef-8b85-3cb72c449250 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.543038 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ecb90-3f30-47ef-8b85-3cb72c449250 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.543725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ecb90-3f30-47ef-8b85-3cb72c449250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:58.545137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ecb90-3f30-47ef-8b85-3cb72c449250 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.556137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 394/791] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jul 11 20:44:58 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:58.556137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ecb90-3f30-47ef-8b85-3cb72c449250 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:58.636322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628580d-5f31-4324-b771-4c9f43ebecea None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:58.710639 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439b81e-2836-4d20-b544-922380383a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:58.711184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ecb90-3f30-47ef-8b85-3cb72c449250 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:58.711911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439b81e-2836-4d20-b544-922380383a97 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.712414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439b81e-2836-4d20-b544-922380383a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.712883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439b81e-2836-4d20-b544-922380383a97 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.713846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439b81e-2836-4d20-b544-922380383a97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:58.714346 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439b81e-2836-4d20-b544-922380383a97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:58.716045 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439b81e-2836-4d20-b544-922380383a97 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.742790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 398/792] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:58.743333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:58.748342 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439b81e-2836-4d20-b544-922380383a97 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:58.749021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6994ecc-89ee-48e4-89c3-2fdb4e8e252a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:58.749948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6994ecc-89ee-48e4-89c3-2fdb4e8e252a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.750223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6994ecc-89ee-48e4-89c3-2fdb4e8e252a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.751103 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6994ecc-89ee-48e4-89c3-2fdb4e8e252a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.768605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6994ecc-89ee-48e4-89c3-2fdb4e8e252a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:58.770109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6994ecc-89ee-48e4-89c3-2fdb4e8e252a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e6994ecc-89ee-48e4-89c3-2fdb4e8e252a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:58.796017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:58.797069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:58.798456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 395/793] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jul 11 20:44:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:58.803759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6994ecc-89ee-48e4-89c3-2fdb4e8e252a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:58.822356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:58.888471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439b81e-2836-4d20-b544-922380383a97 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:58.923328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6994ecc-89ee-48e4-89c3-2fdb4e8e252a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:58.925913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af32b8-0c4b-430e-9ce1-7490bc1b6ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:58.926928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af32b8-0c4b-430e-9ce1-7490bc1b6ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.927302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af32b8-0c4b-430e-9ce1-7490bc1b6ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.927654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af32b8-0c4b-430e-9ce1-7490bc1b6ae0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.937071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af32b8-0c4b-430e-9ce1-7490bc1b6ae0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:58.938642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af32b8-0c4b-430e-9ce1-7490bc1b6ae0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f0af32b8-0c4b-430e-9ce1-7490bc1b6ae0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:44:58.955283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:58.956284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:58.957220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 399/794] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:44:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:44:58.959124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a2ef1-f682-4e57-82c9-4c5e34b35857 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.959473 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a2ef1-f682-4e57-82c9-4c5e34b35857 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.959814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a2ef1-f682-4e57-82c9-4c5e34b35857 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.961452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 400/795] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:58.969561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c8371-d813-4ddb-bf80-03c619d80d90 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.969865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c8371-d813-4ddb-bf80-03c619d80d90 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.970143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c8371-d813-4ddb-bf80-03c619d80d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.971856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77939e7-2455-47c4-aa01-8e59da1d4a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:58.972781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77939e7-2455-47c4-aa01-8e59da1d4a32 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:58.973242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77939e7-2455-47c4-aa01-8e59da1d4a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:58.973743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77939e7-2455-47c4-aa01-8e59da1d4a32 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:58.974784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77939e7-2455-47c4-aa01-8e59da1d4a32 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:58.975287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77939e7-2455-47c4-aa01-8e59da1d4a32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:58.976945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77939e7-2455-47c4-aa01-8e59da1d4a32 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:58.987578 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ac3c8371-d813-4ddb-bf80-03c619d80d90 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:58.988817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ac3c8371-d813-4ddb-bf80-03c619d80d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:58.996527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 396/796] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jul 11 20:44:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:58.997605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77939e7-2455-47c4-aa01-8e59da1d4a32 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:59.052864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:59.069898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 401/797] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:44:59.069898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c8371-d813-4ddb-bf80-03c619d80d90 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:59.108558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:59.150035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77939e7-2455-47c4-aa01-8e59da1d4a32 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:59.185143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c8371-d813-4ddb-bf80-03c619d80d90 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:59.196213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ac7aa-f43f-41d8-835a-b9043cfa92d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:59.197611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac7aa-f43f-41d8-835a-b9043cfa92d8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.198724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac7aa-f43f-41d8-835a-b9043cfa92d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.199201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac7aa-f43f-41d8-835a-b9043cfa92d8 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.200556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ac7aa-f43f-41d8-835a-b9043cfa92d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:59.201469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ac7aa-f43f-41d8-835a-b9043cfa92d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:59.203654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ac7aa-f43f-41d8-835a-b9043cfa92d8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:59.221416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83a8b1-fa3f-4ef3-968a-d5c51c99d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:59.222201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83a8b1-fa3f-4ef3-968a-d5c51c99d84a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.222477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83a8b1-fa3f-4ef3-968a-d5c51c99d84a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.222748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83a8b1-fa3f-4ef3-968a-d5c51c99d84a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.226538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 397/798] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jul 11 20:44:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:59.227560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ac7aa-f43f-41d8-835a-b9043cfa92d8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:59.240777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83a8b1-fa3f-4ef3-968a-d5c51c99d84a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:59.242664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83a8b1-fa3f-4ef3-968a-d5c51c99d84a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-bc83a8b1-fa3f-4ef3-968a-d5c51c99d84a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:59.265907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:59.266859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:59.268205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:59.268205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:59.268205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:59.268205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:59.271986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 402/799] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:59.273805 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83a8b1-fa3f-4ef3-968a-d5c51c99d84a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:59.346280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ac7aa-f43f-41d8-835a-b9043cfa92d8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:59.347844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:59.381593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:59.382440 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17b068-36a8-4aec-bf15-f15ae9ab5e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:59.383615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17b068-36a8-4aec-bf15-f15ae9ab5e4e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.384029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17b068-36a8-4aec-bf15-f15ae9ab5e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.384528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17b068-36a8-4aec-bf15-f15ae9ab5e4e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.385507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17b068-36a8-4aec-bf15-f15ae9ab5e4e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:59.388004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17b068-36a8-4aec-bf15-f15ae9ab5e4e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-bb17b068-36a8-4aec-bf15-f15ae9ab5e4e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:59.401866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:59.402817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 20:44:59.403647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:44:59.407972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83a8b1-fa3f-4ef3-968a-d5c51c99d84a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:59.408130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 398/800] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Jul 11 20:44:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:59.408947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17b068-36a8-4aec-bf15-f15ae9ab5e4e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:59.447586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76693a-8275-4efb-a354-2ef3ba8634c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:59.449245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76693a-8275-4efb-a354-2ef3ba8634c1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.449704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76693a-8275-4efb-a354-2ef3ba8634c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.450178 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76693a-8275-4efb-a354-2ef3ba8634c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.462195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76693a-8275-4efb-a354-2ef3ba8634c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:59.464053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76693a-8275-4efb-a354-2ef3ba8634c1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5a76693a-8275-4efb-a354-2ef3ba8634c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:44:59.484037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:59.485007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:59.485873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:44:59.485873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:44:59.485873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:44:59.485873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:59.487988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 403/801] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:59.488947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76693a-8275-4efb-a354-2ef3ba8634c1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:59.518784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17b068-36a8-4aec-bf15-f15ae9ab5e4e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:59.556436 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f211444-9a46-478c-a097-91cd4f68faa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:59.557506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f211444-9a46-478c-a097-91cd4f68faa7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.557785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f211444-9a46-478c-a097-91cd4f68faa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.558161 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f211444-9a46-478c-a097-91cd4f68faa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.558902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f211444-9a46-478c-a097-91cd4f68faa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:59.559211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f211444-9a46-478c-a097-91cd4f68faa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:59.560780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f211444-9a46-478c-a097-91cd4f68faa7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:59.577508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 399/802] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:59.577508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f211444-9a46-478c-a097-91cd4f68faa7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:59.603944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:59.611482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76693a-8275-4efb-a354-2ef3ba8634c1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:59.645274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9878f38-a68a-423d-980d-21e3a0402f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:59.646146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9878f38-a68a-423d-980d-21e3a0402f53 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.646429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9878f38-a68a-423d-980d-21e3a0402f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.646706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9878f38-a68a-423d-980d-21e3a0402f53 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.647422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9878f38-a68a-423d-980d-21e3a0402f53 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:59.647727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9878f38-a68a-423d-980d-21e3a0402f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:59.649173 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9878f38-a68a-423d-980d-21e3a0402f53 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:59.650480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:59.659585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 404/803] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jul 11 20:44:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:59.659585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c93ef-0c77-4e2c-9c74-aa3bc59db6fc None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.659585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c93ef-0c77-4e2c-9c74-aa3bc59db6fc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.659585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c93ef-0c77-4e2c-9c74-aa3bc59db6fc None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.659585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 405/804] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:44:59.659984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c93ef-0c77-4e2c-9c74-aa3bc59db6fc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:59.690343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f211444-9a46-478c-a097-91cd4f68faa7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:59.728570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999daed-7bb8-4608-94c2-36e78b081155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:59.729275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999daed-7bb8-4608-94c2-36e78b081155 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.729555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999daed-7bb8-4608-94c2-36e78b081155 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.729829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999daed-7bb8-4608-94c2-36e78b081155 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.730559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999daed-7bb8-4608-94c2-36e78b081155 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:59.730864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999daed-7bb8-4608-94c2-36e78b081155 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:59.732269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999daed-7bb8-4608-94c2-36e78b081155 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:59.744618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 400/805] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:59.745737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999daed-7bb8-4608-94c2-36e78b081155 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:59.770307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c93ef-0c77-4e2c-9c74-aa3bc59db6fc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:59.805332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7006f0-a2b4-4aa1-8d88-9f51b0621187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:59.806135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7006f0-a2b4-4aa1-8d88-9f51b0621187 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.806415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7006f0-a2b4-4aa1-8d88-9f51b0621187 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.806706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7006f0-a2b4-4aa1-8d88-9f51b0621187 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.813487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7006f0-a2b4-4aa1-8d88-9f51b0621187 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:59.815152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7006f0-a2b4-4aa1-8d88-9f51b0621187 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-bc7006f0-a2b4-4aa1-8d88-9f51b0621187 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:44:59.831674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:44:59.832820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:44:59.833856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:44:59.833856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:44:59.833856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:44:59.833856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:44:59.833856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:44:59.833856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:44:59.833856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:44:59.833856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:44:59.833856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 406/806] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:44:59.834502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6f866-aaf5-424b-b6ba-1540cfbd4044 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.834785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6f866-aaf5-424b-b6ba-1540cfbd4044 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.835057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6f866-aaf5-424b-b6ba-1540cfbd4044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.847236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:59.855968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3aa6f866-aaf5-424b-b6ba-1540cfbd4044 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:44:59.857207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3aa6f866-aaf5-424b-b6ba-1540cfbd4044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:44:59.865298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999daed-7bb8-4608-94c2-36e78b081155 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:44:59.910028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cfac6-2ed0-4526-ad73-70f157238266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:44:59.911238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cfac6-2ed0-4526-ad73-70f157238266 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:44:59.911689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cfac6-2ed0-4526-ad73-70f157238266 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:44:59.912134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cfac6-2ed0-4526-ad73-70f157238266 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:44:59.913013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cfac6-2ed0-4526-ad73-70f157238266 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:44:59.913503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:44:59.913887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cfac6-2ed0-4526-ad73-70f157238266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:44:59.915593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cfac6-2ed0-4526-ad73-70f157238266 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:44:59.922545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 401/807] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jul 11 20:44:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:44:59.923390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cfac6-2ed0-4526-ad73-70f157238266 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:44:59.944286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 407/808] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:44:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:44:59.945471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6f866-aaf5-424b-b6ba-1540cfbd4044 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:00.031985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cfac6-2ed0-4526-ad73-70f157238266 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:00.058457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6f866-aaf5-424b-b6ba-1540cfbd4044 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:00.066180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5d4f3-f18d-455f-8c12-94b501377504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:00.066993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5d4f3-f18d-455f-8c12-94b501377504 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.067859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5d4f3-f18d-455f-8c12-94b501377504 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.068187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5d4f3-f18d-455f-8c12-94b501377504 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.075442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5d4f3-f18d-455f-8c12-94b501377504 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:00.076928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5d4f3-f18d-455f-8c12-94b501377504 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-89d5d4f3-f18d-455f-8c12-94b501377504 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:00.091431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:00.092566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:00.093457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:00.093457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:00.093457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:00.093457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:00.093457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:00.093457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:00.093457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:00.093457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:00.093457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 402/809] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:00.093457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5d4f3-f18d-455f-8c12-94b501377504 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:00.101053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd97fa-3734-420a-bdda-d6b506632f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:00.101808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd97fa-3734-420a-bdda-d6b506632f44 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.102096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd97fa-3734-420a-bdda-d6b506632f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.102380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd97fa-3734-420a-bdda-d6b506632f44 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.103175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd97fa-3734-420a-bdda-d6b506632f44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:00.103485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd97fa-3734-420a-bdda-d6b506632f44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:00.104964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd97fa-3734-420a-bdda-d6b506632f44 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:00.111277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:00.119072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 408/810] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:44:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:00.119841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd97fa-3734-420a-bdda-d6b506632f44 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:00.169971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:00.212994 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5d4f3-f18d-455f-8c12-94b501377504 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:00.220689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd97fa-3734-420a-bdda-d6b506632f44 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:00.248968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678bd82-5eee-42b6-9dae-57ece8dfefd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:00.250023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678bd82-5eee-42b6-9dae-57ece8dfefd3 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.250479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678bd82-5eee-42b6-9dae-57ece8dfefd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.250955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678bd82-5eee-42b6-9dae-57ece8dfefd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.251867 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678bd82-5eee-42b6-9dae-57ece8dfefd3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:00.253439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678bd82-5eee-42b6-9dae-57ece8dfefd3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:00.254296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917788f-40e2-42c5-b57c-cdaaab95a3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:00.255126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917788f-40e2-42c5-b57c-cdaaab95a3eb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.255494 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917788f-40e2-42c5-b57c-cdaaab95a3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.255770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917788f-40e2-42c5-b57c-cdaaab95a3eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.266112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917788f-40e2-42c5-b57c-cdaaab95a3eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-3917788f-40e2-42c5-b57c-cdaaab95a3eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:00.274936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:00.276349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 409/811] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:45:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:00.277174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917788f-40e2-42c5-b57c-cdaaab95a3eb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:00.281154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 403/812] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jul 11 20:45:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:00.282234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678bd82-5eee-42b6-9dae-57ece8dfefd3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:00.368943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:00.421580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678bd82-5eee-42b6-9dae-57ece8dfefd3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:00.439500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917788f-40e2-42c5-b57c-cdaaab95a3eb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:00.450463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:00.466676 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb550aa-b1d8-4d87-89c8-dab52f3b2a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:00.467702 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb550aa-b1d8-4d87-89c8-dab52f3b2a27 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.467976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb550aa-b1d8-4d87-89c8-dab52f3b2a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.468576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb550aa-b1d8-4d87-89c8-dab52f3b2a27 None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/812f2c4a42fd470c9c57bd14310f320f/roles/91ff1620e1064bf199221383f124aa3c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.486684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb550aa-b1d8-4d87-89c8-dab52f3b2a27 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=812f2c4a42fd470c9c57bd14310f320f, role_id=91ff1620e1064bf199221383f124aa3c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:00.488241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb550aa-b1d8-4d87-89c8-dab52f3b2a27 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:00.488502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce17d6d-7861-4653-9679-4decb91c2d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:00.489612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce17d6d-7861-4653-9679-4decb91c2d40 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.490193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce17d6d-7861-4653-9679-4decb91c2d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.490652 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce17d6d-7861-4653-9679-4decb91c2d40 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.491665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce17d6d-7861-4653-9679-4decb91c2d40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:00.492167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce17d6d-7861-4653-9679-4decb91c2d40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:00.493975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce17d6d-7861-4653-9679-4decb91c2d40 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:00.502363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 404/813] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:45:00 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/812f2c4a42fd470c9c57bd14310f320f/roles/91ff1620e1064bf199221383f124aa3c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:45:00.503408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb550aa-b1d8-4d87-89c8-dab52f3b2a27 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:00.516360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 410/814] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:45:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:00.518544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce17d6d-7861-4653-9679-4decb91c2d40 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:00.677817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb550aa-b1d8-4d87-89c8-dab52f3b2a27 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:00.683561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce17d6d-7861-4653-9679-4decb91c2d40 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:00.700688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:00.732570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086a3b8-8c51-48cc-9b1b-6da9570761a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:00.733887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086a3b8-8c51-48cc-9b1b-6da9570761a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.734182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086a3b8-8c51-48cc-9b1b-6da9570761a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.734460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086a3b8-8c51-48cc-9b1b-6da9570761a2 None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/97b5f28d84fe4570a7c5c904dc4e502c/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.736985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086a3b8-8c51-48cc-9b1b-6da9570761a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=97b5f28d84fe4570a7c5c904dc4e502c, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:00.738456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086a3b8-8c51-48cc-9b1b-6da9570761a2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:00.740631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1140a9b-96d9-4131-b69a-7cd3ba66beb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:00.741585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1140a9b-96d9-4131-b69a-7cd3ba66beb9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.742051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1140a9b-96d9-4131-b69a-7cd3ba66beb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.742507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1140a9b-96d9-4131-b69a-7cd3ba66beb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.743526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1140a9b-96d9-4131-b69a-7cd3ba66beb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:00.743936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1140a9b-96d9-4131-b69a-7cd3ba66beb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:00.747327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1140a9b-96d9-4131-b69a-7cd3ba66beb9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:00.758768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 405/816] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:45:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:00.759043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 411/816] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:45:00 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/97b5f28d84fe4570a7c5c904dc4e502c/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:45:00.764811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1140a9b-96d9-4131-b69a-7cd3ba66beb9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:00.824017 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0e935-2abf-4931-b78d-707c9d14ded1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.824374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0e935-2abf-4931-b78d-707c9d14ded1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.824630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0e935-2abf-4931-b78d-707c9d14ded1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.826336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 412/817] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:00.833732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906cd4c-920a-41d2-99fd-cdfc6538e277 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.834039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906cd4c-920a-41d2-99fd-cdfc6538e277 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.834282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906cd4c-920a-41d2-99fd-cdfc6538e277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.869242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e906cd4c-920a-41d2-99fd-cdfc6538e277 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:00.870470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e906cd4c-920a-41d2-99fd-cdfc6538e277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:00.892724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1140a9b-96d9-4131-b69a-7cd3ba66beb9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:00.924503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:00.945084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248cfeb-50a6-4fd0-af90-2cfa1331432b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:00.946656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248cfeb-50a6-4fd0-af90-2cfa1331432b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:00.947129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248cfeb-50a6-4fd0-af90-2cfa1331432b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:00.947638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248cfeb-50a6-4fd0-af90-2cfa1331432b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:00.966824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248cfeb-50a6-4fd0-af90-2cfa1331432b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:00.968349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248cfeb-50a6-4fd0-af90-2cfa1331432b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-0248cfeb-50a6-4fd0-af90-2cfa1331432b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:00.987056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:00.988011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:00.988968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:00.988968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:00.988968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:00.988968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:00.989781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 406/818] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:00.997121 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248cfeb-50a6-4fd0-af90-2cfa1331432b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:01.001953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 413/819] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:01.030062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906cd4c-920a-41d2-99fd-cdfc6538e277 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:01.050464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:01.191397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906cd4c-920a-41d2-99fd-cdfc6538e277 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:01.193630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248cfeb-50a6-4fd0-af90-2cfa1331432b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:01.247208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5db39-0885-4dc2-b546-41cfda481cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:01.247941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5db39-0885-4dc2-b546-41cfda481cba None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:01.248250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5db39-0885-4dc2-b546-41cfda481cba None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:01.248526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5db39-0885-4dc2-b546-41cfda481cba None admin] PATH_INFO: `/v3/services/network` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:01.249189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5db39-0885-4dc2-b546-41cfda481cba None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:01.250562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5db39-0885-4dc2-b546-41cfda481cba None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:01.259689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff60dc-ac85-493d-9f27-2cc4e8a2ae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:01.260427 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff60dc-ac85-493d-9f27-2cc4e8a2ae68 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:01.260710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff60dc-ac85-493d-9f27-2cc4e8a2ae68 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:01.260988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff60dc-ac85-493d-9f27-2cc4e8a2ae68 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:01.261734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff60dc-ac85-493d-9f27-2cc4e8a2ae68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:01.262041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff60dc-ac85-493d-9f27-2cc4e8a2ae68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:01.263437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff60dc-ac85-493d-9f27-2cc4e8a2ae68 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-afd5db39-0885-4dc2-b546-41cfda481cba None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:01.278932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:01.280299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:01.281214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 414/820] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:01 2022] GET /identity/v3/services/network => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:01.287732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 407/821] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:01.288711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5db39-0885-4dc2-b546-41cfda481cba None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:01.292972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff60dc-ac85-493d-9f27-2cc4e8a2ae68 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:01.339601 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:01.423952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:01.446734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff60dc-ac85-493d-9f27-2cc4e8a2ae68 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:01.486844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473b19-4299-4ac3-84a3-971ef6036df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:01.487792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473b19-4299-4ac3-84a3-971ef6036df6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:01.488074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473b19-4299-4ac3-84a3-971ef6036df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:01.488375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473b19-4299-4ac3-84a3-971ef6036df6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:01.490476 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5db39-0885-4dc2-b546-41cfda481cba None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:01.496523 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23473b19-4299-4ac3-84a3-971ef6036df6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:01.499074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23473b19-4299-4ac3-84a3-971ef6036df6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-23473b19-4299-4ac3-84a3-971ef6036df6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:45:01.517175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:01.518508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:01.519644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 408/822] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jul 11 20:45:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:01.522673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473b19-4299-4ac3-84a3-971ef6036df6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:01.552555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c19644-a2a0-4e42-9ad8-cb8f82301ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:01.553235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c19644-a2a0-4e42-9ad8-cb8f82301ede None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:01.553513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c19644-a2a0-4e42-9ad8-cb8f82301ede None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:01.553836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c19644-a2a0-4e42-9ad8-cb8f82301ede None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:01.554699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c19644-a2a0-4e42-9ad8-cb8f82301ede None admin] RBAC: Adding query filter params (name=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:01.555104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c19644-a2a0-4e42-9ad8-cb8f82301ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:01.556523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c19644-a2a0-4e42-9ad8-cb8f82301ede None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:01.561600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 415/823] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:45:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:01.568403 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c19644-a2a0-4e42-9ad8-cb8f82301ede None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:01.705515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:01.717630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c19644-a2a0-4e42-9ad8-cb8f82301ede None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:01.753636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525267b-bcb8-40dc-8158-d19f35047e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:01.754079 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473b19-4299-4ac3-84a3-971ef6036df6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:01.754939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525267b-bcb8-40dc-8158-d19f35047e83 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:01.755947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525267b-bcb8-40dc-8158-d19f35047e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:01.756462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525267b-bcb8-40dc-8158-d19f35047e83 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:01.757361 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525267b-bcb8-40dc-8158-d19f35047e83 None admin] RBAC: Adding query filter params (type=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:01.757855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525267b-bcb8-40dc-8158-d19f35047e83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:01.759420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525267b-bcb8-40dc-8158-d19f35047e83 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:01.765934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 416/824] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:45:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:01.777042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525267b-bcb8-40dc-8158-d19f35047e83 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:01.796584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4819b4-fdcc-40b4-85f5-55ee7bc1635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:01.797265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4819b4-fdcc-40b4-85f5-55ee7bc1635e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:01.797550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4819b4-fdcc-40b4-85f5-55ee7bc1635e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:01.797822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4819b4-fdcc-40b4-85f5-55ee7bc1635e None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:01.798665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4819b4-fdcc-40b4-85f5-55ee7bc1635e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:01.798971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4819b4-fdcc-40b4-85f5-55ee7bc1635e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:01.812498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4819b4-fdcc-40b4-85f5-55ee7bc1635e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:01.846503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 409/825] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jul 11 20:45:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:01.851985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4819b4-fdcc-40b4-85f5-55ee7bc1635e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:01.860412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:01.927411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525267b-bcb8-40dc-8158-d19f35047e83 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:01.991240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e062c63-562c-4ea1-a722-a966b9177d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:01.991996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e062c63-562c-4ea1-a722-a966b9177d7d None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:01.992312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e062c63-562c-4ea1-a722-a966b9177d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:01.992637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e062c63-562c-4ea1-a722-a966b9177d7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:01.993413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e062c63-562c-4ea1-a722-a966b9177d7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:01.994853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e062c63-562c-4ea1-a722-a966b9177d7d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:02.033112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 417/826] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jul 11 20:45:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:02.039764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4819b4-fdcc-40b4-85f5-55ee7bc1635e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:02.086857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a2e5d-4f85-4aa4-8597-2dc72a91f2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:02.087596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a2e5d-4f85-4aa4-8597-2dc72a91f2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:02.087889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a2e5d-4f85-4aa4-8597-2dc72a91f2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:02.088192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a2e5d-4f85-4aa4-8597-2dc72a91f2d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:02.179961 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:02.183539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a2e5d-4f85-4aa4-8597-2dc72a91f2d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:02.185131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a2e5d-4f85-4aa4-8597-2dc72a91f2d5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:02.304627 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 410/827] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 20:45:02.310062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e062c63-562c-4ea1-a722-a966b9177d7d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:02.320325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:02.425507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e062c63-562c-4ea1-a722-a966b9177d7d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:02.465647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b513a9-c56a-4946-bd30-f93a17d8bb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:02.467275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b513a9-c56a-4946-bd30-f93a17d8bb04 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:02.467949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b513a9-c56a-4946-bd30-f93a17d8bb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:02.468636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b513a9-c56a-4946-bd30-f93a17d8bb04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:02.485880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b513a9-c56a-4946-bd30-f93a17d8bb04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:02.488722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b513a9-c56a-4946-bd30-f93a17d8bb04 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f3b513a9-c56a-4946-bd30-f93a17d8bb04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:02.507850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:02.508874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:02.510750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:02.510750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:02.510750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:02.510750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:02.510750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 418/828] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:02.515583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a2e5d-4f85-4aa4-8597-2dc72a91f2d5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:02.631372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:02.634379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a2e5d-4f85-4aa4-8597-2dc72a91f2d5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:02.670344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537353ed-4ff9-4369-a9c3-09ed6a00554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:02.671022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537353ed-4ff9-4369-a9c3-09ed6a00554f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:02.671303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537353ed-4ff9-4369-a9c3-09ed6a00554f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:02.671571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537353ed-4ff9-4369-a9c3-09ed6a00554f None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:02.672387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537353ed-4ff9-4369-a9c3-09ed6a00554f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:02.672698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537353ed-4ff9-4369-a9c3-09ed6a00554f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:02.674066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537353ed-4ff9-4369-a9c3-09ed6a00554f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:02.686641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 411/829] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:02.697230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b513a9-c56a-4946-bd30-f93a17d8bb04 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:02.720819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60535c-ced7-40c6-843f-bc6e5128c7d6 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:02.721142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60535c-ced7-40c6-843f-bc6e5128c7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:02.721384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60535c-ced7-40c6-843f-bc6e5128c7d6 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:02.723548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 412/830] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:02.728234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf412d4-1a0a-45f3-b9f9-dab5205a8177 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:02.728523 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf412d4-1a0a-45f3-b9f9-dab5205a8177 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:02.728798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf412d4-1a0a-45f3-b9f9-dab5205a8177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:02.747506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4cf412d4-1a0a-45f3-b9f9-dab5205a8177 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:02.748849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4cf412d4-1a0a-45f3-b9f9-dab5205a8177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:02.765721 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:02.810479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b513a9-c56a-4946-bd30-f93a17d8bb04 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:02.854995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb72b7b-7197-41d7-b98d-45f96cad54b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:02.856002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb72b7b-7197-41d7-b98d-45f96cad54b4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:02.856413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb72b7b-7197-41d7-b98d-45f96cad54b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:02.856787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb72b7b-7197-41d7-b98d-45f96cad54b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:02.865190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb72b7b-7197-41d7-b98d-45f96cad54b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:02.866860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb72b7b-7197-41d7-b98d-45f96cad54b4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8eb72b7b-7197-41d7-b98d-45f96cad54b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:02.883078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:02.884340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:02.885457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:02.885457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:02.885457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:02.885457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:02.885457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:02.885457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:02.885457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:02.885457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:02.891283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 419/831] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:02.893249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb72b7b-7197-41d7-b98d-45f96cad54b4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:02.968239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 413/832] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:02.988016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf412d4-1a0a-45f3-b9f9-dab5205a8177 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:03.010279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb72b7b-7197-41d7-b98d-45f96cad54b4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:03.028186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:03.046543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1057ad8-9f8a-4f49-9552-d09641c97668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:03.047217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1057ad8-9f8a-4f49-9552-d09641c97668 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.047493 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1057ad8-9f8a-4f49-9552-d09641c97668 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.047762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1057ad8-9f8a-4f49-9552-d09641c97668 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.048580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1057ad8-9f8a-4f49-9552-d09641c97668 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:03.048885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1057ad8-9f8a-4f49-9552-d09641c97668 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:03.050302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1057ad8-9f8a-4f49-9552-d09641c97668 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:03.062779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 420/833] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:45:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:03.064228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18da6ed-8378-4fb7-82c0-7b20ad25005c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.064531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18da6ed-8378-4fb7-82c0-7b20ad25005c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.064904 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18da6ed-8378-4fb7-82c0-7b20ad25005c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.066127 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 421/834] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:03.068491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18da6ed-8378-4fb7-82c0-7b20ad25005c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:03.094211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:03.104821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf412d4-1a0a-45f3-b9f9-dab5205a8177 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:03.137618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9389a-ccc8-48a0-b8ba-2b7ccf32ae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:03.138343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9389a-ccc8-48a0-b8ba-2b7ccf32ae27 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.138617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9389a-ccc8-48a0-b8ba-2b7ccf32ae27 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.138881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9389a-ccc8-48a0-b8ba-2b7ccf32ae27 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.139528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9389a-ccc8-48a0-b8ba-2b7ccf32ae27 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:03.140953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9389a-ccc8-48a0-b8ba-2b7ccf32ae27 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-90d9389a-ccc8-48a0-b8ba-2b7ccf32ae27 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:03.155407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:03.156489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 20:45:03.157531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:03.161896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 414/835] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:45:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:03.163235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590fa9b-cc12-46ae-9fcf-6947bbf497e5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.163518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590fa9b-cc12-46ae-9fcf-6947bbf497e5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.163792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590fa9b-cc12-46ae-9fcf-6947bbf497e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.180528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d590fa9b-cc12-46ae-9fcf-6947bbf497e5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:03.181933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d590fa9b-cc12-46ae-9fcf-6947bbf497e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:03.194968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18da6ed-8378-4fb7-82c0-7b20ad25005c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:03.231606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4f708-3dcd-40ad-b410-8b4ea5e4f96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:03.232304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4f708-3dcd-40ad-b410-8b4ea5e4f96e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.232572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4f708-3dcd-40ad-b410-8b4ea5e4f96e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.232851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4f708-3dcd-40ad-b410-8b4ea5e4f96e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.233681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4f708-3dcd-40ad-b410-8b4ea5e4f96e None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=acb3ad858e4848b9a325c72c7328c863) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:03.233988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4f708-3dcd-40ad-b410-8b4ea5e4f96e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:03.235412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4f708-3dcd-40ad-b410-8b4ea5e4f96e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:03.241433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 422/836] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:03 2022] GET /identity/v3/role_assignments?user.id=acb3ad858e4848b9a325c72c7328c863&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=60bd1c44ce154338a7face8949d07281 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:03.242174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4f708-3dcd-40ad-b410-8b4ea5e4f96e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:03.257658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 415/837] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:03.259415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7e7da-7da2-4580-8bf0-1237055dd765 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.259545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7e7da-7da2-4580-8bf0-1237055dd765 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.259905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7e7da-7da2-4580-8bf0-1237055dd765 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.261754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 416/838] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:03.269940 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534ff8d-12ea-4a3a-bd26-b61b6e0f90d4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.270322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534ff8d-12ea-4a3a-bd26-b61b6e0f90d4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.270552 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534ff8d-12ea-4a3a-bd26-b61b6e0f90d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.296781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0534ff8d-12ea-4a3a-bd26-b61b6e0f90d4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:03.298234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0534ff8d-12ea-4a3a-bd26-b61b6e0f90d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:03.307826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:03.373000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4f708-3dcd-40ad-b410-8b4ea5e4f96e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:03.384281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 417/839] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:03.385061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534ff8d-12ea-4a3a-bd26-b61b6e0f90d4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:03.404815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:03.412458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13595400-c6fd-4c10-99d3-e436e72ced54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:03.413280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595400-c6fd-4c10-99d3-e436e72ced54 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.413614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595400-c6fd-4c10-99d3-e436e72ced54 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.413993 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595400-c6fd-4c10-99d3-e436e72ced54 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.414773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13595400-c6fd-4c10-99d3-e436e72ced54 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:03.415143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13595400-c6fd-4c10-99d3-e436e72ced54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:03.416596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13595400-c6fd-4c10-99d3-e436e72ced54 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:03.422354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 423/840] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:45:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:03.423195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13595400-c6fd-4c10-99d3-e436e72ced54 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:03.511498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534ff8d-12ea-4a3a-bd26-b61b6e0f90d4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:03.546347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13595400-c6fd-4c10-99d3-e436e72ced54 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:03.556017 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:03.566468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2be97-b6ea-4056-b6a6-37ef29444064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:03.568137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2be97-b6ea-4056-b6a6-37ef29444064 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.568492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2be97-b6ea-4056-b6a6-37ef29444064 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.568733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2be97-b6ea-4056-b6a6-37ef29444064 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.580454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2be97-b6ea-4056-b6a6-37ef29444064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-92c2be97-b6ea-4056-b6a6-37ef29444064 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:03.588986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:03.589901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:03.590728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 418/841] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:45:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:03.591400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2be97-b6ea-4056-b6a6-37ef29444064 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:03.592370 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a1db6-f542-454d-8aa4-e284fea2e767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:03.593179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a1db6-f542-454d-8aa4-e284fea2e767 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.593457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a1db6-f542-454d-8aa4-e284fea2e767 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.593726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a1db6-f542-454d-8aa4-e284fea2e767 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.605885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a1db6-f542-454d-8aa4-e284fea2e767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-321a1db6-f542-454d-8aa4-e284fea2e767 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:03.613258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:03.614561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:03.615390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 424/842] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:45:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:03.616175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a1db6-f542-454d-8aa4-e284fea2e767 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:03.685219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:03.714524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2be97-b6ea-4056-b6a6-37ef29444064 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:03.726094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a1db6-f542-454d-8aa4-e284fea2e767 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:03.751625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede49029-c548-4c84-a003-5740832698b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:03.752335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede49029-c548-4c84-a003-5740832698b7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.752621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede49029-c548-4c84-a003-5740832698b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.752904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede49029-c548-4c84-a003-5740832698b7 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.753942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede49029-c548-4c84-a003-5740832698b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:03.754252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede49029-c548-4c84-a003-5740832698b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:03.756432 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede49029-c548-4c84-a003-5740832698b7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:03.760837 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe2e9c-faaa-4f19-a41a-3a4e6a8689b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:03.761425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 419/843] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:45:03 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:03.761547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe2e9c-faaa-4f19-a41a-3a4e6a8689b0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.761780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe2e9c-faaa-4f19-a41a-3a4e6a8689b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.762054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe2e9c-faaa-4f19-a41a-3a4e6a8689b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.762808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe2e9c-faaa-4f19-a41a-3a4e6a8689b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:03.763119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe2e9c-faaa-4f19-a41a-3a4e6a8689b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:03.764556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe2e9c-faaa-4f19-a41a-3a4e6a8689b0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:03.771983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede49029-c548-4c84-a003-5740832698b7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:03.779317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 425/844] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:45:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:03.780515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe2e9c-faaa-4f19-a41a-3a4e6a8689b0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:03.813531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:03.900632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe2e9c-faaa-4f19-a41a-3a4e6a8689b0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:03.918550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede49029-c548-4c84-a003-5740832698b7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:03.938740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ce456-d700-4035-ad7a-df029d394ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:03.939856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ce456-d700-4035-ad7a-df029d394ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.939856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ce456-d700-4035-ad7a-df029d394ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.940141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ce456-d700-4035-ad7a-df029d394ec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.957492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ce456-d700-4035-ad7a-df029d394ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:03.960005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ce456-d700-4035-ad7a-df029d394ec1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:03.969411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c5286-4a1c-4433-86b1-4bb96f5368ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:03.970405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c5286-4a1c-4433-86b1-4bb96f5368ae None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:03.970820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c5286-4a1c-4433-86b1-4bb96f5368ae None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:03.971224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c5286-4a1c-4433-86b1-4bb96f5368ae None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:03.972167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c5286-4a1c-4433-86b1-4bb96f5368ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:03.972467 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c5286-4a1c-4433-86b1-4bb96f5368ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:03.974097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c5286-4a1c-4433-86b1-4bb96f5368ae None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6a7ce456-d700-4035-ad7a-df029d394ec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:03.980666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:03.981539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:03.982430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:03.982430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:03.982430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:03.982430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:03.983097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 426/845] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:03.983644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:03.983865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 420/846] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:03.988372 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ce456-d700-4035-ad7a-df029d394ec1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:03.990560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c5286-4a1c-4433-86b1-4bb96f5368ae None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:04.088157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:04.106834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c5286-4a1c-4433-86b1-4bb96f5368ae None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:04.109661 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ce456-d700-4035-ad7a-df029d394ec1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:04.145853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d63e3-6e26-4822-b048-6a6713919191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:04.146984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d63e3-6e26-4822-b048-6a6713919191 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:04.147584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d63e3-6e26-4822-b048-6a6713919191 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:04.148157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d63e3-6e26-4822-b048-6a6713919191 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:04.148676 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4dddc-47d2-43c9-b8db-777878f846f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:04.149269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d63e3-6e26-4822-b048-6a6713919191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:04.149483 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4dddc-47d2-43c9-b8db-777878f846f7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:04.149761 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4dddc-47d2-43c9-b8db-777878f846f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:04.150028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4dddc-47d2-43c9-b8db-777878f846f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:04.150274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d63e3-6e26-4822-b048-6a6713919191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:04.152043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d63e3-6e26-4822-b048-6a6713919191 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:04.163003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4dddc-47d2-43c9-b8db-777878f846f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:04.164476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4dddc-47d2-43c9-b8db-777878f846f7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:04.165524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 427/847] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:04.171457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d63e3-6e26-4822-b048-6a6713919191 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b3b4dddc-47d2-43c9-b8db-777878f846f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:04.188476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:04.189631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:04.190730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:04.190730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:04.190730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:04.190730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:04.191823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 421/848] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:04.197251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4dddc-47d2-43c9-b8db-777878f846f7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:04.256507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:04.295810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d63e3-6e26-4822-b048-6a6713919191 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:04.312433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4dddc-47d2-43c9-b8db-777878f846f7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:04.348448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcf2a7-fd43-4100-8003-edaa8cf92db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:04.349840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcf2a7-fd43-4100-8003-edaa8cf92db8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:04.350654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcf2a7-fd43-4100-8003-edaa8cf92db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:04.351427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcf2a7-fd43-4100-8003-edaa8cf92db8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:04.353543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5aeffd-302a-426e-a1b3-919e643ce639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:04.355309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5aeffd-302a-426e-a1b3-919e643ce639 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:04.355740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5aeffd-302a-426e-a1b3-919e643ce639 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:04.356201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5aeffd-302a-426e-a1b3-919e643ce639 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:04.357087 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5aeffd-302a-426e-a1b3-919e643ce639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:04.357533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5aeffd-302a-426e-a1b3-919e643ce639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:04.359330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5aeffd-302a-426e-a1b3-919e643ce639 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:04.360037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcf2a7-fd43-4100-8003-edaa8cf92db8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:04.361563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcf2a7-fd43-4100-8003-edaa8cf92db8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:04.364887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:04.375594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 422/849] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:04.382597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5aeffd-302a-426e-a1b3-919e643ce639 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ccbcf2a7-fd43-4100-8003-edaa8cf92db8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:45:04.384181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:04.385609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:04.386774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:04.386774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:04.386774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:45:04.386774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:45:04.386774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:45:04.386774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:45:04.386774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:45:04.386774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:45:04.386774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:45:04.386774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:04.393603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 428/850] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jul 11 20:45:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:04.394669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcf2a7-fd43-4100-8003-edaa8cf92db8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:04.502618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5aeffd-302a-426e-a1b3-919e643ce639 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:04.517910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:04.535643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcf2a7-fd43-4100-8003-edaa8cf92db8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:04.548482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1042643-3259-4525-b463-4437193b41ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:04.549498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1042643-3259-4525-b463-4437193b41ad None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:04.550166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1042643-3259-4525-b463-4437193b41ad None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:04.550578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1042643-3259-4525-b463-4437193b41ad None admin] PATH_INFO: `/v3/users/swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:04.561209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1042643-3259-4525-b463-4437193b41ad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:04.563377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1042643-3259-4525-b463-4437193b41ad None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:04.571825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b9123-3109-48da-963b-ebcdc7506614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:04.572655 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b9123-3109-48da-963b-ebcdc7506614 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:04.573508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b9123-3109-48da-963b-ebcdc7506614 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:04.573783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b9123-3109-48da-963b-ebcdc7506614 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:04.574803 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b9123-3109-48da-963b-ebcdc7506614 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:04.575204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b9123-3109-48da-963b-ebcdc7506614 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:04.576778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b9123-3109-48da-963b-ebcdc7506614 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b1042643-3259-4525-b463-4437193b41ad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:45:04.584979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:04.586100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:04.587020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:04.587020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:04.587020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:45:04.587020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:45:04.587020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:45:04.587020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:45:04.587020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:45:04.587020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:45:04.587020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 20:45:04.587020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:04.591660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 423/851] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:45:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:04.593076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1042643-3259-4525-b463-4437193b41ad None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:04.611101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 429/852] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jul 11 20:45:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:04.612731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b9123-3109-48da-963b-ebcdc7506614 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:04.689374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:04.712260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1042643-3259-4525-b463-4437193b41ad None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:04.733446 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b9123-3109-48da-963b-ebcdc7506614 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:04.750583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83e158-cbe9-493b-a9f9-9d27917bf617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:04.751549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83e158-cbe9-493b-a9f9-9d27917bf617 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:04.751969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83e158-cbe9-493b-a9f9-9d27917bf617 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:04.752423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83e158-cbe9-493b-a9f9-9d27917bf617 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:04.753464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83e158-cbe9-493b-a9f9-9d27917bf617 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:04.753914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83e158-cbe9-493b-a9f9-9d27917bf617 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:04.755753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83e158-cbe9-493b-a9f9-9d27917bf617 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:04.769356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94612d89-f1e6-474b-b161-b3d68580effb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:04.770191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94612d89-f1e6-474b-b161-b3d68580effb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:04.770573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94612d89-f1e6-474b-b161-b3d68580effb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:04.771029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94612d89-f1e6-474b-b161-b3d68580effb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:04.777209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 424/853] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jul 11 20:45:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:04.785147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83e158-cbe9-493b-a9f9-9d27917bf617 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:04.818189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:04.854564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94612d89-f1e6-474b-b161-b3d68580effb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:04.856017 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94612d89-f1e6-474b-b161-b3d68580effb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:04.898828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83e158-cbe9-493b-a9f9-9d27917bf617 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:04.914732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:04.950060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ca28f-0b69-464c-980a-496f83c682ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:04.950782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ca28f-0b69-464c-980a-496f83c682ce None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:04.951043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ca28f-0b69-464c-980a-496f83c682ce None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:04.951318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ca28f-0b69-464c-980a-496f83c682ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:04.979836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 430/854] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:04 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:45:04.981647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94612d89-f1e6-474b-b161-b3d68580effb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:05.050152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ca28f-0b69-464c-980a-496f83c682ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:05.051847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ca28f-0b69-464c-980a-496f83c682ce None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:05.088936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:05.113762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94612d89-f1e6-474b-b161-b3d68580effb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:05.157613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97024f9-d9eb-4508-8aec-53496d0559f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:05.158682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97024f9-d9eb-4508-8aec-53496d0559f8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.159114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97024f9-d9eb-4508-8aec-53496d0559f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.159528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97024f9-d9eb-4508-8aec-53496d0559f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.170928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97024f9-d9eb-4508-8aec-53496d0559f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:05.172430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97024f9-d9eb-4508-8aec-53496d0559f8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:05.183964 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 425/855] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:04 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 20:45:05.192290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ca28f-0b69-464c-980a-496f83c682ce None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d97024f9-d9eb-4508-8aec-53496d0559f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:05.192442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:05.193318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:05.194171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:05.194171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:05.194171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:05.194171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:05.194171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 431/856] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:05.196891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97024f9-d9eb-4508-8aec-53496d0559f8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:05.214379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:05.313173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ca28f-0b69-464c-980a-496f83c682ce None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:05.322035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97024f9-d9eb-4508-8aec-53496d0559f8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:05.345307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:05.352986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa66c1-1abd-4c28-ac45-02d668050d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:05.354021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa66c1-1abd-4c28-ac45-02d668050d19 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.354436 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa66c1-1abd-4c28-ac45-02d668050d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.354847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa66c1-1abd-4c28-ac45-02d668050d19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.358181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb1ab0-4ee3-422f-a6d4-a89c3fec7faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:05.358856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb1ab0-4ee3-422f-a6d4-a89c3fec7faf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.359139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb1ab0-4ee3-422f-a6d4-a89c3fec7faf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.359412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb1ab0-4ee3-422f-a6d4-a89c3fec7faf None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.360188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb1ab0-4ee3-422f-a6d4-a89c3fec7faf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:05.360501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb1ab0-4ee3-422f-a6d4-a89c3fec7faf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:05.361961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb1ab0-4ee3-422f-a6d4-a89c3fec7faf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:05.367417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa66c1-1abd-4c28-ac45-02d668050d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:05.369197 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa66c1-1abd-4c28-ac45-02d668050d19 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:05.374419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 432/857] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:05.375863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d5466-8005-4707-9c68-81124311dc02 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.376193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d5466-8005-4707-9c68-81124311dc02 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.376474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d5466-8005-4707-9c68-81124311dc02 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.377942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 433/858] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:05.380319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d5466-8005-4707-9c68-81124311dc02 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c2aa66c1-1abd-4c28-ac45-02d668050d19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:05.388673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:05.389756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:05.390819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:05.390819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:05.390819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:05.390819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:05.390819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 426/859] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:05.393337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d748d-06f2-449e-9e07-357cc49f8c4c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.393337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d748d-06f2-449e-9e07-357cc49f8c4c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.393337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d748d-06f2-449e-9e07-357cc49f8c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.412289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f76d748d-06f2-449e-9e07-357cc49f8c4c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:05.413566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f76d748d-06f2-449e-9e07-357cc49f8c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:05.492248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 427/860] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:05.492248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d748d-06f2-449e-9e07-357cc49f8c4c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:05.580412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:05.606023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d748d-06f2-449e-9e07-357cc49f8c4c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:05.614363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d5466-8005-4707-9c68-81124311dc02 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:05.621048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:05.644678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db996533-fde3-49c7-bb49-8ef7fcd2ff7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:05.645474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db996533-fde3-49c7-bb49-8ef7fcd2ff7f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.645754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db996533-fde3-49c7-bb49-8ef7fcd2ff7f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.646028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db996533-fde3-49c7-bb49-8ef7fcd2ff7f None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.646754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db996533-fde3-49c7-bb49-8ef7fcd2ff7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:05.647058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db996533-fde3-49c7-bb49-8ef7fcd2ff7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:05.648456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db996533-fde3-49c7-bb49-8ef7fcd2ff7f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:05.650248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42289005-2851-41fd-b540-78868df4eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:05.651026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42289005-2851-41fd-b540-78868df4eab4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.651309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42289005-2851-41fd-b540-78868df4eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.651579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42289005-2851-41fd-b540-78868df4eab4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.658287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42289005-2851-41fd-b540-78868df4eab4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:05.659733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42289005-2851-41fd-b540-78868df4eab4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:05.661041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 428/861] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:05.662579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db996533-fde3-49c7-bb49-8ef7fcd2ff7f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-42289005-2851-41fd-b540-78868df4eab4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:05.674069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:05.675130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:05.676147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:05.676147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:05.676147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:05.676147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:05.676147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:05.676147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:05.676147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:05.676147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:05.676147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 434/862] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:05.676778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42289005-2851-41fd-b540-78868df4eab4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:05.774243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db996533-fde3-49c7-bb49-8ef7fcd2ff7f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:05.780077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42289005-2851-41fd-b540-78868df4eab4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:05.809626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:05.809815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82971e03-eea1-4479-a67d-99104c2d7f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:05.810625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82971e03-eea1-4479-a67d-99104c2d7f64 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.810902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82971e03-eea1-4479-a67d-99104c2d7f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.811181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82971e03-eea1-4479-a67d-99104c2d7f64 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.811836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82971e03-eea1-4479-a67d-99104c2d7f64 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:05.813344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82971e03-eea1-4479-a67d-99104c2d7f64 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:05.817511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125c397-d285-4801-a87a-a01e6ba6de0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:05.818431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125c397-d285-4801-a87a-a01e6ba6de0d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.818716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125c397-d285-4801-a87a-a01e6ba6de0d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.819004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125c397-d285-4801-a87a-a01e6ba6de0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.826745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125c397-d285-4801-a87a-a01e6ba6de0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:05.828222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125c397-d285-4801-a87a-a01e6ba6de0d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-82971e03-eea1-4479-a67d-99104c2d7f64 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:05.832623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:05.833800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 20:45:05.834641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:05.838869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 429/863] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Jul 11 20:45:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:05.839648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82971e03-eea1-4479-a67d-99104c2d7f64 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1125c397-d285-4801-a87a-a01e6ba6de0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:05.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:05.846148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:05.852087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:05.852087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:05.852087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:05.852087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:05.852087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:05.852087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:05.852087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:05.852087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:05.852087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 435/864] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:05.852087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125c397-d285-4801-a87a-a01e6ba6de0d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:05.879113 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:05.942474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82971e03-eea1-4479-a67d-99104c2d7f64 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:05.949725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125c397-d285-4801-a87a-a01e6ba6de0d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:05.977622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171b2bc-3be0-4919-921b-5f5811c97c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:05.978317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171b2bc-3be0-4919-921b-5f5811c97c9c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.978636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171b2bc-3be0-4919-921b-5f5811c97c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.978914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171b2bc-3be0-4919-921b-5f5811c97c9c None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.979704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171b2bc-3be0-4919-921b-5f5811c97c9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:05.980010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171b2bc-3be0-4919-921b-5f5811c97c9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:05.981484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171b2bc-3be0-4919-921b-5f5811c97c9c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:05.983785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e05b8-eb58-4ebf-9902-a79e7c2a674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:05.984470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e05b8-eb58-4ebf-9902-a79e7c2a674a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.984749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e05b8-eb58-4ebf-9902-a79e7c2a674a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.985023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e05b8-eb58-4ebf-9902-a79e7c2a674a None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.985720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e05b8-eb58-4ebf-9902-a79e7c2a674a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:05.986023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e05b8-eb58-4ebf-9902-a79e7c2a674a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:05.987367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e05b8-eb58-4ebf-9902-a79e7c2a674a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:05.992682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 436/865] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:05.993442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e05b8-eb58-4ebf-9902-a79e7c2a674a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:05.993835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 430/866] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:05.995086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76869883-8ec8-4e1c-b77e-4c89f1669b48 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:05.995369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76869883-8ec8-4e1c-b77e-4c89f1669b48 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:05.995642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76869883-8ec8-4e1c-b77e-4c89f1669b48 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:05.997036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 431/867] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:05.998963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76869883-8ec8-4e1c-b77e-4c89f1669b48 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:06.054176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:06.094928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:06.098686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e05b8-eb58-4ebf-9902-a79e7c2a674a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:06.102433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76869883-8ec8-4e1c-b77e-4c89f1669b48 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:06.131635 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3d5bd-5556-48cf-b10e-33cd7d2f8758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:06.132327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3d5bd-5556-48cf-b10e-33cd7d2f8758 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:06.132606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3d5bd-5556-48cf-b10e-33cd7d2f8758 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:06.132872 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3d5bd-5556-48cf-b10e-33cd7d2f8758 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:06.133647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3d5bd-5556-48cf-b10e-33cd7d2f8758 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:06.133949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3d5bd-5556-48cf-b10e-33cd7d2f8758 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:06.135367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3d5bd-5556-48cf-b10e-33cd7d2f8758 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:06.136237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a24da-97b4-4653-b5b0-33e25b48e8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:06.136995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a24da-97b4-4653-b5b0-33e25b48e8db None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:06.137366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a24da-97b4-4653-b5b0-33e25b48e8db None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:06.137736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a24da-97b4-4653-b5b0-33e25b48e8db None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:06.138451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a24da-97b4-4653-b5b0-33e25b48e8db None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:06.138756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a24da-97b4-4653-b5b0-33e25b48e8db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:06.140155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a24da-97b4-4653-b5b0-33e25b48e8db None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:06.144851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 432/868] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:06.145665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a24da-97b4-4653-b5b0-33e25b48e8db None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:06.149738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 437/869] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:06.151255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc3477-5d2f-4fbd-a4c4-277d5280b2c7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:06.151698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc3477-5d2f-4fbd-a4c4-277d5280b2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:06.151984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc3477-5d2f-4fbd-a4c4-277d5280b2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:06.169949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-15dc3477-5d2f-4fbd-a4c4-277d5280b2c7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:06.171130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-15dc3477-5d2f-4fbd-a4c4-277d5280b2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:06.248176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 438/870] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:06.248176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc3477-5d2f-4fbd-a4c4-277d5280b2c7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:06.265987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a24da-97b4-4653-b5b0-33e25b48e8db None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:06.304359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce53243-7475-425e-8a8c-23f6c224d06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:06.305153 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce53243-7475-425e-8a8c-23f6c224d06a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:06.305749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce53243-7475-425e-8a8c-23f6c224d06a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:06.306036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce53243-7475-425e-8a8c-23f6c224d06a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:06.307155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce53243-7475-425e-8a8c-23f6c224d06a None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=812f2c4a42fd470c9c57bd14310f320f) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:06.307469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce53243-7475-425e-8a8c-23f6c224d06a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:06.308038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:06.309528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce53243-7475-425e-8a8c-23f6c224d06a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:06.316157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 433/871] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:06 2022] GET /identity/v3/role_assignments?user.id=812f2c4a42fd470c9c57bd14310f320f&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:06.317160 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce53243-7475-425e-8a8c-23f6c224d06a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:06.346528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:06.389122 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc3477-5d2f-4fbd-a4c4-277d5280b2c7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:06.444315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2571f2c-1250-4122-85c1-46d04df0468d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:06.445006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2571f2c-1250-4122-85c1-46d04df0468d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:06.445287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2571f2c-1250-4122-85c1-46d04df0468d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:06.445561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2571f2c-1250-4122-85c1-46d04df0468d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:06.446431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2571f2c-1250-4122-85c1-46d04df0468d None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=97b5f28d84fe4570a7c5c904dc4e502c) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:06.446739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2571f2c-1250-4122-85c1-46d04df0468d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:06.448649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2571f2c-1250-4122-85c1-46d04df0468d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:06.455846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce53243-7475-425e-8a8c-23f6c224d06a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:06.464133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 439/872] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:06 2022] GET /identity/v3/role_assignments?user.id=97b5f28d84fe4570a7c5c904dc4e502c&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=60bd1c44ce154338a7face8949d07281 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:06.508263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b1458-5001-4c15-a7c2-52e0b7348887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:06.509191 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1458-5001-4c15-a7c2-52e0b7348887 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:06.509579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1458-5001-4c15-a7c2-52e0b7348887 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:06.510018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1458-5001-4c15-a7c2-52e0b7348887 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:06.510848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b1458-5001-4c15-a7c2-52e0b7348887 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:06.512474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b1458-5001-4c15-a7c2-52e0b7348887 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:06.531552 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 434/873] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:45:06 2022] POST /identity/v3/services => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:08.231289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7231ebb-53aa-4187-913e-9b026d3114af None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.231820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7231ebb-53aa-4187-913e-9b026d3114af None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.231820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7231ebb-53aa-4187-913e-9b026d3114af None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.233486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 440/874] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:08.239015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3c655-ae32-4e4a-94f5-cd96c56b0801 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.239511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3c655-ae32-4e4a-94f5-cd96c56b0801 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.239592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3c655-ae32-4e4a-94f5-cd96c56b0801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.257280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a4a3c655-ae32-4e4a-94f5-cd96c56b0801 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:08.258460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a4a3c655-ae32-4e4a-94f5-cd96c56b0801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:08.339036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 435/875] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:08 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:08.361045 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7231ebb-53aa-4187-913e-9b026d3114af None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:08.467535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7231ebb-53aa-4187-913e-9b026d3114af None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:08.478255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:08.504491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4101687-40d1-4bdf-8187-8bf94ca14eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:08.505439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4101687-40d1-4bdf-8187-8bf94ca14eb1 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.505886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4101687-40d1-4bdf-8187-8bf94ca14eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.506299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4101687-40d1-4bdf-8187-8bf94ca14eb1 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.507183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4101687-40d1-4bdf-8187-8bf94ca14eb1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:08.508952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4101687-40d1-4bdf-8187-8bf94ca14eb1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:08.521960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafe9e4-a119-478d-b373-dc69635e09cb None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.522289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafe9e4-a119-478d-b373-dc69635e09cb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.522507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafe9e4-a119-478d-b373-dc69635e09cb None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.524016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 436/876] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:08.532038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 441/877] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:45:08 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:08.534842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af53f6d-02b7-4308-bc11-4f356caaea5d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.535356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af53f6d-02b7-4308-bc11-4f356caaea5d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.535959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af53f6d-02b7-4308-bc11-4f356caaea5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.555024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2af53f6d-02b7-4308-bc11-4f356caaea5d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:08.556711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2af53f6d-02b7-4308-bc11-4f356caaea5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:08.560616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:08.577090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9106-e76b-4b3c-8fac-30b83bae7e4d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.577439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9106-e76b-4b3c-8fac-30b83bae7e4d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.577637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9106-e76b-4b3c-8fac-30b83bae7e4d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.579166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 442/878] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:08.585319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e99d8a-8041-416e-918c-e0cf4cf06cf1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.585946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e99d8a-8041-416e-918c-e0cf4cf06cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.586536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e99d8a-8041-416e-918c-e0cf4cf06cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.605681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a6e99d8a-8041-416e-918c-e0cf4cf06cf1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:08.607201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a6e99d8a-8041-416e-918c-e0cf4cf06cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:08.653837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 437/879] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:08.678099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c0b83-4c21-4c16-8c6d-6cf352432249 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.678462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c0b83-4c21-4c16-8c6d-6cf352432249 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.678638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c0b83-4c21-4c16-8c6d-6cf352432249 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.680047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 438/880] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:08.681674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c0b83-4c21-4c16-8c6d-6cf352432249 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:08.695771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 443/881] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:08.697227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb21a0a-35e9-4f34-a6a8-5a5283e540a9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.697431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb21a0a-35e9-4f34-a6a8-5a5283e540a9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.697696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb21a0a-35e9-4f34-a6a8-5a5283e540a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.715346 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6fb21a0a-35e9-4f34-a6a8-5a5283e540a9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:08.716912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6fb21a0a-35e9-4f34-a6a8-5a5283e540a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:08.789371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 444/882] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:08.790216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb21a0a-35e9-4f34-a6a8-5a5283e540a9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:08.819510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:08.821399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c0b83-4c21-4c16-8c6d-6cf352432249 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:08.862128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c9288-b00b-44db-8a9d-d1a8369ef16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:08.862947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c9288-b00b-44db-8a9d-d1a8369ef16f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.863226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c9288-b00b-44db-8a9d-d1a8369ef16f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.863511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c9288-b00b-44db-8a9d-d1a8369ef16f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.866268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:08.875343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c9288-b00b-44db-8a9d-d1a8369ef16f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-812c9288-b00b-44db-8a9d-d1a8369ef16f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:08.883796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:08.884781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:08.885606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 439/883] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jul 11 20:45:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:08.887014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c9288-b00b-44db-8a9d-d1a8369ef16f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:08.907613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb21a0a-35e9-4f34-a6a8-5a5283e540a9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:08.943514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ff317-f787-4f8e-91bf-3e9ea037956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:08.944368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff317-f787-4f8e-91bf-3e9ea037956e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:08.944655 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff317-f787-4f8e-91bf-3e9ea037956e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:08.944925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff317-f787-4f8e-91bf-3e9ea037956e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:08.945589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ff317-f787-4f8e-91bf-3e9ea037956e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:08.946963 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ff317-f787-4f8e-91bf-3e9ea037956e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ac2ff317-f787-4f8e-91bf-3e9ea037956e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:08.959800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:08.960949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:08.961793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 445/884] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:08.962593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ff317-f787-4f8e-91bf-3e9ea037956e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.012839 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c9288-b00b-44db-8a9d-d1a8369ef16f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.052746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435272a-5e68-41d3-a85f-4e2cc82ea09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:09.053881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435272a-5e68-41d3-a85f-4e2cc82ea09d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:09.054160 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435272a-5e68-41d3-a85f-4e2cc82ea09d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:09.054445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435272a-5e68-41d3-a85f-4e2cc82ea09d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:09.055114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435272a-5e68-41d3-a85f-4e2cc82ea09d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:09.056527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435272a-5e68-41d3-a85f-4e2cc82ea09d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:09.068469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-4435272a-5e68-41d3-a85f-4e2cc82ea09d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:09.069979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:09.071597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:09.072720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 440/885] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:45:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:09.073923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435272a-5e68-41d3-a85f-4e2cc82ea09d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.085942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ff317-f787-4f8e-91bf-3e9ea037956e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.124292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc2c6d-47ab-466f-ae87-702acf0772b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:09.124968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc2c6d-47ab-466f-ae87-702acf0772b0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:09.125244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc2c6d-47ab-466f-ae87-702acf0772b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:09.125688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc2c6d-47ab-466f-ae87-702acf0772b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:09.126990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc2c6d-47ab-466f-ae87-702acf0772b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:09.127479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc2c6d-47ab-466f-ae87-702acf0772b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:09.130170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc2c6d-47ab-466f-ae87-702acf0772b0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:09.142949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 446/886] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jul 11 20:45:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:09.143746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc2c6d-47ab-466f-ae87-702acf0772b0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.150799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:09.199002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435272a-5e68-41d3-a85f-4e2cc82ea09d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.234543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e2a9a-df27-413b-92ac-915d42f599da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:09.235240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2a9a-df27-413b-92ac-915d42f599da None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:09.235518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2a9a-df27-413b-92ac-915d42f599da None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:09.235787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2a9a-df27-413b-92ac-915d42f599da None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:09.236526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e2a9a-df27-413b-92ac-915d42f599da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:09.236833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e2a9a-df27-413b-92ac-915d42f599da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:09.238204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e2a9a-df27-413b-92ac-915d42f599da None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:09.243023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 441/887] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jul 11 20:45:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:09.243791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e2a9a-df27-413b-92ac-915d42f599da None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.265576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc2c6d-47ab-466f-ae87-702acf0772b0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.302352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23660508-47c7-426d-92e8-856b011ff0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:09.303079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23660508-47c7-426d-92e8-856b011ff0ef None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:09.303357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23660508-47c7-426d-92e8-856b011ff0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:09.303632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23660508-47c7-426d-92e8-856b011ff0ef None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:09.304390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23660508-47c7-426d-92e8-856b011ff0ef None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:09.304695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23660508-47c7-426d-92e8-856b011ff0ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:09.306092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23660508-47c7-426d-92e8-856b011ff0ef None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:09.311684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 447/888] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:45:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:09.312500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23660508-47c7-426d-92e8-856b011ff0ef None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.333109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:09.355252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e2a9a-df27-413b-92ac-915d42f599da None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.378370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:09.390966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e2f80-45b9-40ae-bb46-9844d32aebbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:09.391761 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e2f80-45b9-40ae-bb46-9844d32aebbf None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:09.392037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e2f80-45b9-40ae-bb46-9844d32aebbf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:09.392346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e2f80-45b9-40ae-bb46-9844d32aebbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:09.404286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e2f80-45b9-40ae-bb46-9844d32aebbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:09.405725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e2f80-45b9-40ae-bb46-9844d32aebbf None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:09.418753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23660508-47c7-426d-92e8-856b011ff0ef None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c64e2f80-45b9-40ae-bb46-9844d32aebbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:09.422736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:09.423618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:09.424587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:09.424587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:09.424587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:09.424587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:09.424587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 442/889] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:09.424861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e2f80-45b9-40ae-bb46-9844d32aebbf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.455934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfca72-ef04-452b-ada9-e264cd7f0a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:09.456652 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfca72-ef04-452b-ada9-e264cd7f0a51 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:09.456977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfca72-ef04-452b-ada9-e264cd7f0a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:09.457239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfca72-ef04-452b-ada9-e264cd7f0a51 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:09.458051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfca72-ef04-452b-ada9-e264cd7f0a51 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:09.458228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfca72-ef04-452b-ada9-e264cd7f0a51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:09.459724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfca72-ef04-452b-ada9-e264cd7f0a51 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:09.476144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 448/890] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jul 11 20:45:09 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:09.476144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfca72-ef04-452b-ada9-e264cd7f0a51 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.572208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e2f80-45b9-40ae-bb46-9844d32aebbf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.584642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:09.609342 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfca72-ef04-452b-ada9-e264cd7f0a51 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.622929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f9b22-8495-4d66-9ebf-9c59b6cfdefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:09.624284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f9b22-8495-4d66-9ebf-9c59b6cfdefe None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:09.624764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f9b22-8495-4d66-9ebf-9c59b6cfdefe None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:09.625736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f9b22-8495-4d66-9ebf-9c59b6cfdefe None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:09.626966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f9b22-8495-4d66-9ebf-9c59b6cfdefe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:09.627520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f9b22-8495-4d66-9ebf-9c59b6cfdefe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:09.629825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f9b22-8495-4d66-9ebf-9c59b6cfdefe None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:09.637816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 443/891] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:45:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:09.644472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f9b22-8495-4d66-9ebf-9c59b6cfdefe None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.653173 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31054dd9-0670-45b6-af1f-508dad269523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:09.653859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31054dd9-0670-45b6-af1f-508dad269523 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:09.654142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31054dd9-0670-45b6-af1f-508dad269523 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:09.654417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31054dd9-0670-45b6-af1f-508dad269523 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:09.655153 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31054dd9-0670-45b6-af1f-508dad269523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:09.655463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31054dd9-0670-45b6-af1f-508dad269523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:09.657625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31054dd9-0670-45b6-af1f-508dad269523 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:09.670462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 449/892] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:09.677009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31054dd9-0670-45b6-af1f-508dad269523 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.713671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:09.781057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f9b22-8495-4d66-9ebf-9c59b6cfdefe None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.797399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31054dd9-0670-45b6-af1f-508dad269523 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.828125 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fabe36-3299-4ab9-a5ff-deb257bed338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:09.829031 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fabe36-3299-4ab9-a5ff-deb257bed338 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:09.829320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fabe36-3299-4ab9-a5ff-deb257bed338 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:09.829592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fabe36-3299-4ab9-a5ff-deb257bed338 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:09.830662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fabe36-3299-4ab9-a5ff-deb257bed338 None admin] RBAC: Adding query filter params (interface=public, service_id=43be67dfe27a4f3691d645801aee9de5, region_id=RegionOne) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:09.833121 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fabe36-3299-4ab9-a5ff-deb257bed338 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:09.834811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fabe36-3299-4ab9-a5ff-deb257bed338 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:09.835506 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8520d1-b9a6-406a-b4cf-3bb3e5ca3f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:09.836643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8520d1-b9a6-406a-b4cf-3bb3e5ca3f72 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:09.837126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8520d1-b9a6-406a-b4cf-3bb3e5ca3f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:09.837600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8520d1-b9a6-406a-b4cf-3bb3e5ca3f72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:09.841739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 444/893] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jul 11 20:45:09 2022] GET /identity/v3/endpoints?service_id=43be67dfe27a4f3691d645801aee9de5&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:09.848142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fabe36-3299-4ab9-a5ff-deb257bed338 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.849238 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8520d1-b9a6-406a-b4cf-3bb3e5ca3f72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:09.850713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8520d1-b9a6-406a-b4cf-3bb3e5ca3f72 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7b8520d1-b9a6-406a-b4cf-3bb3e5ca3f72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:45:09.866880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:09.867857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:09.873771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 450/894] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jul 11 20:45:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:09.876244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8520d1-b9a6-406a-b4cf-3bb3e5ca3f72 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:09.880526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:09.975813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fabe36-3299-4ab9-a5ff-deb257bed338 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:09.997953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:10.001894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8520d1-b9a6-406a-b4cf-3bb3e5ca3f72 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:10.012539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e0ce8-83a7-4c72-bb7c-478548230fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:10.013605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0ce8-83a7-4c72-bb7c-478548230fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:10.014131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0ce8-83a7-4c72-bb7c-478548230fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:10.014600 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0ce8-83a7-4c72-bb7c-478548230fc2 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:10.015532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e0ce8-83a7-4c72-bb7c-478548230fc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:10.017387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e0ce8-83a7-4c72-bb7c-478548230fc2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:10.028663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 445/895] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jul 11 20:45:09 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:45:10.038068 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4abf26-ac58-4a74-b9b0-c7800c5e6880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:10.038840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4abf26-ac58-4a74-b9b0-c7800c5e6880 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:10.039126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4abf26-ac58-4a74-b9b0-c7800c5e6880 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:10.039400 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4abf26-ac58-4a74-b9b0-c7800c5e6880 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:10.040285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4abf26-ac58-4a74-b9b0-c7800c5e6880 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:10.040593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4abf26-ac58-4a74-b9b0-c7800c5e6880 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:10.042063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4abf26-ac58-4a74-b9b0-c7800c5e6880 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:10.068258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 451/896] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jul 11 20:45:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:10.069949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e0ce8-83a7-4c72-bb7c-478548230fc2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:10.184251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e0ce8-83a7-4c72-bb7c-478548230fc2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:10.221303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff6843-8f08-469a-957d-0381be9064c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:10.221968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff6843-8f08-469a-957d-0381be9064c6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:10.222249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff6843-8f08-469a-957d-0381be9064c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:10.222530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff6843-8f08-469a-957d-0381be9064c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:10.260657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:10.296341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff6843-8f08-469a-957d-0381be9064c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:10.298159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff6843-8f08-469a-957d-0381be9064c6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:10.399548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 446/897] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:10 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:45:10.405183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4abf26-ac58-4a74-b9b0-c7800c5e6880 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:10.465808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:10.510694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4abf26-ac58-4a74-b9b0-c7800c5e6880 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:10.558498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1df137-3810-42b2-b621-b43b3a767c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:10.559303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1df137-3810-42b2-b621-b43b3a767c1c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:10.559583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1df137-3810-42b2-b621-b43b3a767c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:10.559853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1df137-3810-42b2-b621-b43b3a767c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:10.570408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1df137-3810-42b2-b621-b43b3a767c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:10.571854 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1df137-3810-42b2-b621-b43b3a767c1c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8d1df137-3810-42b2-b621-b43b3a767c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:10.589969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:10.590988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:10.591856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:10.591856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:10.591856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:10.591856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:10.595891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 452/898] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:10.597337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff6843-8f08-469a-957d-0381be9064c6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:10.609776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7492d6f-3308-4564-bacf-ea5afdfa274d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:10.610063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7492d6f-3308-4564-bacf-ea5afdfa274d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:10.610322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7492d6f-3308-4564-bacf-ea5afdfa274d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:10.611811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 453/899] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:10.617310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba468-979f-4b4e-9dea-b4d4c337d266 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:10.617849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba468-979f-4b4e-9dea-b4d4c337d266 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:10.618328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba468-979f-4b4e-9dea-b4d4c337d266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:10.638648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f93ba468-979f-4b4e-9dea-b4d4c337d266 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:10.639850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f93ba468-979f-4b4e-9dea-b4d4c337d266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:10.671110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:10.705728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff6843-8f08-469a-957d-0381be9064c6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:10.723775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 454/900] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:10.749555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba468-979f-4b4e-9dea-b4d4c337d266 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:10.754012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb423c-8d8b-419a-8710-6238411b8b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:10.754818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb423c-8d8b-419a-8710-6238411b8b07 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:10.755094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb423c-8d8b-419a-8710-6238411b8b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:10.755360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb423c-8d8b-419a-8710-6238411b8b07 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:10.756183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb423c-8d8b-419a-8710-6238411b8b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:10.756537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb423c-8d8b-419a-8710-6238411b8b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:10.757918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb423c-8d8b-419a-8710-6238411b8b07 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:10.774470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 447/901] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:10.775798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb423c-8d8b-419a-8710-6238411b8b07 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:10.802811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:10.878217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba468-979f-4b4e-9dea-b4d4c337d266 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:10.894395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb423c-8d8b-419a-8710-6238411b8b07 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:10.914096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8acf7-bff4-4aae-9f01-200c9e459367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:10.915212 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8acf7-bff4-4aae-9f01-200c9e459367 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:10.915694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8acf7-bff4-4aae-9f01-200c9e459367 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:10.916220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8acf7-bff4-4aae-9f01-200c9e459367 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:10.917099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8acf7-bff4-4aae-9f01-200c9e459367 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:10.918675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8acf7-bff4-4aae-9f01-200c9e459367 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:10.921513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:10.931003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a83981-2a6e-4023-84a8-f96ca21edd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:10.932004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a83981-2a6e-4023-84a8-f96ca21edd07 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:10.932447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a83981-2a6e-4023-84a8-f96ca21edd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:10.932855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a83981-2a6e-4023-84a8-f96ca21edd07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-4ee8acf7-bff4-4aae-9f01-200c9e459367 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:10.933196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:10.934299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 20:45:10.935333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:10.942704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 455/902] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Jul 11 20:45:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:10.943684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8acf7-bff4-4aae-9f01-200c9e459367 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:10.943903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a83981-2a6e-4023-84a8-f96ca21edd07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:10.945518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a83981-2a6e-4023-84a8-f96ca21edd07 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-62a83981-2a6e-4023-84a8-f96ca21edd07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:10.961053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:10.961952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:10.962826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:10.962826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:10.962826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:10.962826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:10.962826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:10.962826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:10.962826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:10.962826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:10.965290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 448/903] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:10.974020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a83981-2a6e-4023-84a8-f96ca21edd07 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:11.046495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8acf7-bff4-4aae-9f01-200c9e459367 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:11.053811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:11.079461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a83981-2a6e-4023-84a8-f96ca21edd07 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:11.083367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7502a-5236-4169-bb45-3d8f71249aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:11.084207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7502a-5236-4169-bb45-3d8f71249aef None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:11.084562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7502a-5236-4169-bb45-3d8f71249aef None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:11.084928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7502a-5236-4169-bb45-3d8f71249aef None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:11.085718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7502a-5236-4169-bb45-3d8f71249aef None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:11.086094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7502a-5236-4169-bb45-3d8f71249aef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:11.087558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7502a-5236-4169-bb45-3d8f71249aef None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:11.097348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 456/904] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:11.098326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7502a-5236-4169-bb45-3d8f71249aef None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:11.118661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550357d-e6eb-4055-8e2f-8d0ac713c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:11.119654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550357d-e6eb-4055-8e2f-8d0ac713c91e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:11.120123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550357d-e6eb-4055-8e2f-8d0ac713c91e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:11.120560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550357d-e6eb-4055-8e2f-8d0ac713c91e None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:11.121515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550357d-e6eb-4055-8e2f-8d0ac713c91e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:11.122069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550357d-e6eb-4055-8e2f-8d0ac713c91e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:11.123679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550357d-e6eb-4055-8e2f-8d0ac713c91e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:11.142170 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 449/905] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:11.143836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550357d-e6eb-4055-8e2f-8d0ac713c91e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:11.196986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:11.202724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7502a-5236-4169-bb45-3d8f71249aef None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:11.245205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01004ef0-6cc0-4c44-ad57-5ec877ebbcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:11.246099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01004ef0-6cc0-4c44-ad57-5ec877ebbcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:11.246390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01004ef0-6cc0-4c44-ad57-5ec877ebbcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:11.246668 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01004ef0-6cc0-4c44-ad57-5ec877ebbcc4 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:11.248038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01004ef0-6cc0-4c44-ad57-5ec877ebbcc4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:11.248387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01004ef0-6cc0-4c44-ad57-5ec877ebbcc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:11.250114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01004ef0-6cc0-4c44-ad57-5ec877ebbcc4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:11.250651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550357d-e6eb-4055-8e2f-8d0ac713c91e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:11.260270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 457/906] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:11.263674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01004ef0-6cc0-4c44-ad57-5ec877ebbcc4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:11.298293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e18d5-d156-40a2-80d1-e8de10c2a4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:11.299016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e18d5-d156-40a2-80d1-e8de10c2a4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:11.299356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e18d5-d156-40a2-80d1-e8de10c2a4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:11.299792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e18d5-d156-40a2-80d1-e8de10c2a4b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:11.300724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e18d5-d156-40a2-80d1-e8de10c2a4b2 None admin] RBAC: Adding query filter params (role.id=f37ae457d604457bbe6e5dd28ce225e2, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=812f2c4a42fd470c9c57bd14310f320f) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:11.301043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e18d5-d156-40a2-80d1-e8de10c2a4b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:11.302788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e18d5-d156-40a2-80d1-e8de10c2a4b2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:11.306620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:11.316281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 450/907] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:11 2022] GET /identity/v3/role_assignments?user.id=812f2c4a42fd470c9c57bd14310f320f&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=f37ae457d604457bbe6e5dd28ce225e2 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:11.381755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01004ef0-6cc0-4c44-ad57-5ec877ebbcc4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:11.418620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805f710-7e5d-4966-9f85-af7a2357149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:11.420241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805f710-7e5d-4966-9f85-af7a2357149f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:11.420731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805f710-7e5d-4966-9f85-af7a2357149f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:11.421203 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805f710-7e5d-4966-9f85-af7a2357149f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:11.422215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805f710-7e5d-4966-9f85-af7a2357149f None admin] RBAC: Adding query filter params (interface=public, service_id=3bfe6e9f95f44be897510a90069ee378, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:11.422732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805f710-7e5d-4966-9f85-af7a2357149f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:11.424390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805f710-7e5d-4966-9f85-af7a2357149f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:11.431100 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 458/908] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jul 11 20:45:11 2022] GET /identity/v3/endpoints?service_id=3bfe6e9f95f44be897510a90069ee378&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:11.437383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e18d5-d156-40a2-80d1-e8de10c2a4b2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:11.544581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e18d5-d156-40a2-80d1-e8de10c2a4b2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:11.558793 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:11.580097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dc64e-8e94-4239-8930-f076d6c8e946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:11.580814 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dc64e-8e94-4239-8930-f076d6c8e946 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:11.581099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dc64e-8e94-4239-8930-f076d6c8e946 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:11.581377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dc64e-8e94-4239-8930-f076d6c8e946 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:11.582076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dc64e-8e94-4239-8930-f076d6c8e946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:11.583573 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dc64e-8e94-4239-8930-f076d6c8e946 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:11.589777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 451/909] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jul 11 20:45:11 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:45:11.618254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beb8a6-c41d-4904-a1da-7d348802e3f2 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:11.618544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beb8a6-c41d-4904-a1da-7d348802e3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:11.618828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beb8a6-c41d-4904-a1da-7d348802e3f2 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:11.620240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 459/910] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:11.633466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd03750-8c34-44b2-8994-d0c7d6754c20 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:11.633791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd03750-8c34-44b2-8994-d0c7d6754c20 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:11.634092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd03750-8c34-44b2-8994-d0c7d6754c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:11.652634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-acd03750-8c34-44b2-8994-d0c7d6754c20 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:11.653876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-acd03750-8c34-44b2-8994-d0c7d6754c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:11.727553 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 452/911] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:11.750636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38beb8a6-c41d-4904-a1da-7d348802e3f2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:11.869925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:11.874971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38beb8a6-c41d-4904-a1da-7d348802e3f2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:11.910842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71997cf9-da01-4245-a3d1-17a377824122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:11.911515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71997cf9-da01-4245-a3d1-17a377824122 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:11.911793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71997cf9-da01-4245-a3d1-17a377824122 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:11.912064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71997cf9-da01-4245-a3d1-17a377824122 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:11.912826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71997cf9-da01-4245-a3d1-17a377824122 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:11.914309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71997cf9-da01-4245-a3d1-17a377824122 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:11.931448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 460/912] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:45:11 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:12.254843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ebff9-6022-4eee-82a2-4a4591b9f2e1 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:12.255210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ebff9-6022-4eee-82a2-4a4591b9f2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:12.255491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ebff9-6022-4eee-82a2-4a4591b9f2e1 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:12.257081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 453/913] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:12.262338 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5600cc-f547-4e41-b6d6-af4050ce2300 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:12.262628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5600cc-f547-4e41-b6d6-af4050ce2300 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:12.262903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5600cc-f547-4e41-b6d6-af4050ce2300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:12.279882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0e5600cc-f547-4e41-b6d6-af4050ce2300 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:12.281109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0e5600cc-f547-4e41-b6d6-af4050ce2300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:12.361615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 461/914] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:12.384747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ebff9-6022-4eee-82a2-4a4591b9f2e1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:12.495701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ebff9-6022-4eee-82a2-4a4591b9f2e1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:12.503308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:12.530848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e48a1a-3729-4e9a-8e64-88d1b79629dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:12.531642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e48a1a-3729-4e9a-8e64-88d1b79629dd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:12.531926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e48a1a-3729-4e9a-8e64-88d1b79629dd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:12.532297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e48a1a-3729-4e9a-8e64-88d1b79629dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:12.533015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e48a1a-3729-4e9a-8e64-88d1b79629dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:12.534401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e48a1a-3729-4e9a-8e64-88d1b79629dd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b7e48a1a-3729-4e9a-8e64-88d1b79629dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:12.546617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:12.547555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:12.548411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 454/915] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:45:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:12.553868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5600cc-f547-4e41-b6d6-af4050ce2300 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:12.636471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:12.655251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5600cc-f547-4e41-b6d6-af4050ce2300 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:12.689060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a512-385c-466f-8607-e7b0c1c7ee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:12.689734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a512-385c-466f-8607-e7b0c1c7ee9d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:12.690016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a512-385c-466f-8607-e7b0c1c7ee9d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:12.690292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a512-385c-466f-8607-e7b0c1c7ee9d None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:12.691116 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a512-385c-466f-8607-e7b0c1c7ee9d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:12.691422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a512-385c-466f-8607-e7b0c1c7ee9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:12.693296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a512-385c-466f-8607-e7b0c1c7ee9d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:12.710773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 462/916] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:45:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:12.716463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e48a1a-3729-4e9a-8e64-88d1b79629dd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:12.824221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e48a1a-3729-4e9a-8e64-88d1b79629dd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:12.858243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8f86e-e5de-49ce-9761-4b20653da21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:12.858919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f86e-e5de-49ce-9761-4b20653da21c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:12.859199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f86e-e5de-49ce-9761-4b20653da21c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:12.859473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f86e-e5de-49ce-9761-4b20653da21c None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:12.860222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f86e-e5de-49ce-9761-4b20653da21c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:12.860530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f86e-e5de-49ce-9761-4b20653da21c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:12.861910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f86e-e5de-49ce-9761-4b20653da21c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:12.868137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 455/917] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jul 11 20:45:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:12.873769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a512-385c-466f-8607-e7b0c1c7ee9d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:12.973498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a512-385c-466f-8607-e7b0c1c7ee9d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:13.008202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db37f4-f9bc-40c5-81bd-afa63b2597ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:13.008873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db37f4-f9bc-40c5-81bd-afa63b2597ed None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:13.009155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db37f4-f9bc-40c5-81bd-afa63b2597ed None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:13.009432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db37f4-f9bc-40c5-81bd-afa63b2597ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:13.010186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db37f4-f9bc-40c5-81bd-afa63b2597ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:13.011564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db37f4-f9bc-40c5-81bd-afa63b2597ed None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:13.031719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 463/918] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jul 11 20:45:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:13.420548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490a4b5-33db-4095-97bc-04b48e88ecad None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:13.421075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490a4b5-33db-4095-97bc-04b48e88ecad None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:13.421075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490a4b5-33db-4095-97bc-04b48e88ecad None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:13.422715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 456/919] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:13.428238 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fdcb1-97bd-403c-8b68-cf0e1ed299e3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:13.428810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fdcb1-97bd-403c-8b68-cf0e1ed299e3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:13.429283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fdcb1-97bd-403c-8b68-cf0e1ed299e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:13.435616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:13.447408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e62fdcb1-97bd-403c-8b68-cf0e1ed299e3 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:13.449306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e62fdcb1-97bd-403c-8b68-cf0e1ed299e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:13.529893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 464/920] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:13.552488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490a4b5-33db-4095-97bc-04b48e88ecad None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:13.590272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:13.659059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490a4b5-33db-4095-97bc-04b48e88ecad None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:13.693816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a620b-8b26-4cb4-bbd4-80321f30fd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:13.694623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a620b-8b26-4cb4-bbd4-80321f30fd6d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:13.694906 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a620b-8b26-4cb4-bbd4-80321f30fd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:13.695190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a620b-8b26-4cb4-bbd4-80321f30fd6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:13.706737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a620b-8b26-4cb4-bbd4-80321f30fd6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-1a4a620b-8b26-4cb4-bbd4-80321f30fd6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:13.715083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:13.716065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:13.716951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 457/921] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jul 11 20:45:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:13.722890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fdcb1-97bd-403c-8b68-cf0e1ed299e3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:13.764396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1c281-3aa0-44ae-9676-2fabcfea6c05 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:13.764714 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1c281-3aa0-44ae-9676-2fabcfea6c05 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:13.764951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1c281-3aa0-44ae-9676-2fabcfea6c05 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:13.766721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 458/922] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:13.771421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b64af3-1d3f-4550-82ce-3b6ff78a5df0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:13.771691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b64af3-1d3f-4550-82ce-3b6ff78a5df0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:13.771965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b64af3-1d3f-4550-82ce-3b6ff78a5df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:13.790167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-52b64af3-1d3f-4550-82ce-3b6ff78a5df0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:13.791191 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-52b64af3-1d3f-4550-82ce-3b6ff78a5df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:13.836559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fdcb1-97bd-403c-8b68-cf0e1ed299e3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:13.866204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:13.873761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfd8ad-02da-4cee-b819-76f17a175f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:13.874420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfd8ad-02da-4cee-b819-76f17a175f6c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:13.874702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfd8ad-02da-4cee-b819-76f17a175f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:13.874968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfd8ad-02da-4cee-b819-76f17a175f6c None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:13.875732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfd8ad-02da-4cee-b819-76f17a175f6c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:13.876030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfd8ad-02da-4cee-b819-76f17a175f6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:13.877520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfd8ad-02da-4cee-b819-76f17a175f6c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:13.888156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 459/923] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:13.892120 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 465/924] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jul 11 20:45:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:13.897928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b64af3-1d3f-4550-82ce-3b6ff78a5df0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:13.907561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfd8ad-02da-4cee-b819-76f17a175f6c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:13.960584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:14.021280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b64af3-1d3f-4550-82ce-3b6ff78a5df0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.030817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfd8ad-02da-4cee-b819-76f17a175f6c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.064682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ab44e-6103-40a8-8ecd-31d5dedc3b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.065482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab44e-6103-40a8-8ecd-31d5dedc3b97 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.065762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab44e-6103-40a8-8ecd-31d5dedc3b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.066106 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab44e-6103-40a8-8ecd-31d5dedc3b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.067887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df68c99-e3a1-40c0-84fc-6b83a3c80ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.069302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df68c99-e3a1-40c0-84fc-6b83a3c80ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.069598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df68c99-e3a1-40c0-84fc-6b83a3c80ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.069875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df68c99-e3a1-40c0-84fc-6b83a3c80ef1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.070541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df68c99-e3a1-40c0-84fc-6b83a3c80ef1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.071923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df68c99-e3a1-40c0-84fc-6b83a3c80ef1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:14.076926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ab44e-6103-40a8-8ecd-31d5dedc3b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.078361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ab44e-6103-40a8-8ecd-31d5dedc3b97 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9df68c99-e3a1-40c0-84fc-6b83a3c80ef1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:14.085985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:14.087034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:14.087925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 466/925] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Jul 11 20:45:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:14.088889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900d658-5810-436a-b682-6ada873550e0 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.088889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900d658-5810-436a-b682-6ada873550e0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.089018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900d658-5810-436a-b682-6ada873550e0 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.094196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 467/926] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:14.094964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900d658-5810-436a-b682-6ada873550e0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-1a9ab44e-6103-40a8-8ecd-31d5dedc3b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:14.102285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:14.103341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:14.104383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:14.104383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:14.104383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:14.104383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:14.106186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 460/927] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:14.107579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b050b-cfa9-4d5a-a292-e93641a47076 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.107863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b050b-cfa9-4d5a-a292-e93641a47076 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.108157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b050b-cfa9-4d5a-a292-e93641a47076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.125376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-951b050b-cfa9-4d5a-a292-e93641a47076 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:14.126549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-951b050b-cfa9-4d5a-a292-e93641a47076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:14.156458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:14.197105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 461/928] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:14.198357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b050b-cfa9-4d5a-a292-e93641a47076 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:14.214384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:14.221013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900d658-5810-436a-b682-6ada873550e0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.259136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201f20-85ec-4ee7-b5ca-2f7a28e192db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.259813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201f20-85ec-4ee7-b5ca-2f7a28e192db None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.260097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201f20-85ec-4ee7-b5ca-2f7a28e192db None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.260395 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201f20-85ec-4ee7-b5ca-2f7a28e192db None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.261239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab201f20-85ec-4ee7-b5ca-2f7a28e192db None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:14.261668 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab201f20-85ec-4ee7-b5ca-2f7a28e192db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.263029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab201f20-85ec-4ee7-b5ca-2f7a28e192db None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:14.273366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 468/929] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:14.273366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201f20-85ec-4ee7-b5ca-2f7a28e192db None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:14.308742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b050b-cfa9-4d5a-a292-e93641a47076 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.348841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcbf7a-de01-43fe-8942-fb0aade1d3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.349510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcbf7a-de01-43fe-8942-fb0aade1d3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.349783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcbf7a-de01-43fe-8942-fb0aade1d3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.350051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcbf7a-de01-43fe-8942-fb0aade1d3d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.350772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcbf7a-de01-43fe-8942-fb0aade1d3d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:14.351072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcbf7a-de01-43fe-8942-fb0aade1d3d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.352462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcbf7a-de01-43fe-8942-fb0aade1d3d8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:14.363729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 462/930] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:14.364574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcbf7a-de01-43fe-8942-fb0aade1d3d8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:14.391420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201f20-85ec-4ee7-b5ca-2f7a28e192db None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.412778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:14.429397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86430b3f-a52e-4dd1-b4e3-fd9ffc8adcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.430534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86430b3f-a52e-4dd1-b4e3-fd9ffc8adcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.430813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86430b3f-a52e-4dd1-b4e3-fd9ffc8adcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.431094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86430b3f-a52e-4dd1-b4e3-fd9ffc8adcb4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.431756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86430b3f-a52e-4dd1-b4e3-fd9ffc8adcb4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.433174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86430b3f-a52e-4dd1-b4e3-fd9ffc8adcb4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:14.433751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-86430b3f-a52e-4dd1-b4e3-fd9ffc8adcb4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:14.445901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:14.446969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:14.448271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 469/931] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:14.450097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86430b3f-a52e-4dd1-b4e3-fd9ffc8adcb4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:14.480025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcbf7a-de01-43fe-8942-fb0aade1d3d8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.514051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d4d21-bc21-49c2-87f5-866546e6d9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.514726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d4d21-bc21-49c2-87f5-866546e6d9dd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.515008 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d4d21-bc21-49c2-87f5-866546e6d9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.515292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d4d21-bc21-49c2-87f5-866546e6d9dd None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.516003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d4d21-bc21-49c2-87f5-866546e6d9dd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:14.516342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d4d21-bc21-49c2-87f5-866546e6d9dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.517723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d4d21-bc21-49c2-87f5-866546e6d9dd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:14.522732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 463/932] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:14.523524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d4d21-bc21-49c2-87f5-866546e6d9dd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:14.562056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86430b3f-a52e-4dd1-b4e3-fd9ffc8adcb4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.601353 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946eec5-6b81-4ada-8095-6007fa590be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.602197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946eec5-6b81-4ada-8095-6007fa590be4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.602862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946eec5-6b81-4ada-8095-6007fa590be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.603142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946eec5-6b81-4ada-8095-6007fa590be4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.612466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946eec5-6b81-4ada-8095-6007fa590be4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.614316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946eec5-6b81-4ada-8095-6007fa590be4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:14.627117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d4d21-bc21-49c2-87f5-866546e6d9dd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c946eec5-6b81-4ada-8095-6007fa590be4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:45:14.629357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:14.630467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 470/933] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jul 11 20:45:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:14.631452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946eec5-6b81-4ada-8095-6007fa590be4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:14.665309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:14.672838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c29372-aa46-4d32-9d97-90f8e7f3c946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.673530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29372-aa46-4d32-9d97-90f8e7f3c946 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.673832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29372-aa46-4d32-9d97-90f8e7f3c946 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.674089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29372-aa46-4d32-9d97-90f8e7f3c946 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.674811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c29372-aa46-4d32-9d97-90f8e7f3c946 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:14.675123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c29372-aa46-4d32-9d97-90f8e7f3c946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.677369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c29372-aa46-4d32-9d97-90f8e7f3c946 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:14.678931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:14.682889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 464/934] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jul 11 20:45:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:14.683774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c29372-aa46-4d32-9d97-90f8e7f3c946 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:14.745073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946eec5-6b81-4ada-8095-6007fa590be4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.782151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64f0e5-2efa-409e-acfa-463bb50dc417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.782810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64f0e5-2efa-409e-acfa-463bb50dc417 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.783097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64f0e5-2efa-409e-acfa-463bb50dc417 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.783387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64f0e5-2efa-409e-acfa-463bb50dc417 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.784140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64f0e5-2efa-409e-acfa-463bb50dc417 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.785648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64f0e5-2efa-409e-acfa-463bb50dc417 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:14.792739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c29372-aa46-4d32-9d97-90f8e7f3c946 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.808247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 471/935] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jul 11 20:45:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:14.809088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64f0e5-2efa-409e-acfa-463bb50dc417 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:14.852099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad148cc-2d43-4b96-a8f5-f269b4e1fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.853295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad148cc-2d43-4b96-a8f5-f269b4e1fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.853982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad148cc-2d43-4b96-a8f5-f269b4e1fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.854537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad148cc-2d43-4b96-a8f5-f269b4e1fcd4 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.855755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad148cc-2d43-4b96-a8f5-f269b4e1fcd4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:14.856323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad148cc-2d43-4b96-a8f5-f269b4e1fcd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.858185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad148cc-2d43-4b96-a8f5-f269b4e1fcd4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:14.881187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 465/936] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jul 11 20:45:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:14.886668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad148cc-2d43-4b96-a8f5-f269b4e1fcd4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:14.936530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:14.944398 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64f0e5-2efa-409e-acfa-463bb50dc417 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:14.961228 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:14.984548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc64e7-04bd-43c2-8d9b-e1fb6cebdd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:14.985597 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc64e7-04bd-43c2-8d9b-e1fb6cebdd6c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:14.986083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc64e7-04bd-43c2-8d9b-e1fb6cebdd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:14.986560 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc64e7-04bd-43c2-8d9b-e1fb6cebdd6c None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:14.987731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc64e7-04bd-43c2-8d9b-e1fb6cebdd6c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:14.988292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc64e7-04bd-43c2-8d9b-e1fb6cebdd6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:14.989946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc64e7-04bd-43c2-8d9b-e1fb6cebdd6c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:15.000497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 472/937] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jul 11 20:45:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:15.002310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc64e7-04bd-43c2-8d9b-e1fb6cebdd6c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:15.019795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad148cc-2d43-4b96-a8f5-f269b4e1fcd4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:15.064755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5a62c-b9ef-4f17-9eb9-15c88a0141c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:15.065579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5a62c-b9ef-4f17-9eb9-15c88a0141c9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.065863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5a62c-b9ef-4f17-9eb9-15c88a0141c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.066143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5a62c-b9ef-4f17-9eb9-15c88a0141c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.082637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5a62c-b9ef-4f17-9eb9-15c88a0141c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:15.084558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5a62c-b9ef-4f17-9eb9-15c88a0141c9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9ab5a62c-b9ef-4f17-9eb9-15c88a0141c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:15.108768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:15.109721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:15.110592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:15.110592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:15.110592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:15.110592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:15.113452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 466/938] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:15.119921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5a62c-b9ef-4f17-9eb9-15c88a0141c9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:15.130834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc64e7-04bd-43c2-8d9b-e1fb6cebdd6c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:15.166891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3056c-88a4-4b7d-b3c0-0bca99df7057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:15.167562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3056c-88a4-4b7d-b3c0-0bca99df7057 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.167844 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3056c-88a4-4b7d-b3c0-0bca99df7057 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.168276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3056c-88a4-4b7d-b3c0-0bca99df7057 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.169074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3056c-88a4-4b7d-b3c0-0bca99df7057 None admin] RBAC: Adding query filter params (interface=public, service_id=b3026f959b5149e8b3919365729883fa, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:15.169385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3056c-88a4-4b7d-b3c0-0bca99df7057 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:15.170876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3056c-88a4-4b7d-b3c0-0bca99df7057 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:15.180752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 473/939] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jul 11 20:45:15 2022] GET /identity/v3/endpoints?service_id=b3026f959b5149e8b3919365729883fa&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:15.180752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40822ff-bce7-46ba-80c8-41c7ab2c2af5 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.180752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40822ff-bce7-46ba-80c8-41c7ab2c2af5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.180752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40822ff-bce7-46ba-80c8-41c7ab2c2af5 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.180752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 474/940] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:15.184470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40822ff-bce7-46ba-80c8-41c7ab2c2af5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:15.217480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:15.236719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:15.240862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5a62c-b9ef-4f17-9eb9-15c88a0141c9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:15.287544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e09c0-0c70-4b77-9b1e-73d25be4f174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:15.288154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e09c0-0c70-4b77-9b1e-73d25be4f174 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.288448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e09c0-0c70-4b77-9b1e-73d25be4f174 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.288709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e09c0-0c70-4b77-9b1e-73d25be4f174 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.289447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e09c0-0c70-4b77-9b1e-73d25be4f174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:15.289756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e09c0-0c70-4b77-9b1e-73d25be4f174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:15.291142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e09c0-0c70-4b77-9b1e-73d25be4f174 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:15.297299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40822ff-bce7-46ba-80c8-41c7ab2c2af5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:15.307730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 467/941] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:15.307730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c9c76-34d8-4601-9023-ee062c5195e8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.307730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c9c76-34d8-4601-9023-ee062c5195e8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.307730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c9c76-34d8-4601-9023-ee062c5195e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.322057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-440c9c76-34d8-4601-9023-ee062c5195e8 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:15.323592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-440c9c76-34d8-4601-9023-ee062c5195e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:15.340081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c75a6-3cf3-474b-a5ba-2305ff5b6059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:15.340786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c75a6-3cf3-474b-a5ba-2305ff5b6059 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.341069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c75a6-3cf3-474b-a5ba-2305ff5b6059 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.341344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c75a6-3cf3-474b-a5ba-2305ff5b6059 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.342035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c75a6-3cf3-474b-a5ba-2305ff5b6059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:15.343414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c75a6-3cf3-474b-a5ba-2305ff5b6059 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:15.351112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 475/942] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jul 11 20:45:15 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:45:15.351112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c75a6-3cf3-474b-a5ba-2305ff5b6059 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:15.404730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 468/943] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:15.423375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c9c76-34d8-4601-9023-ee062c5195e8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:15.493434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:15.507169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c75a6-3cf3-474b-a5ba-2305ff5b6059 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:15.534063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:15.549561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c9c76-34d8-4601-9023-ee062c5195e8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:15.553799 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54de7fa-867c-4e4d-bc90-1c605e7c4f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:15.554209 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54de7fa-867c-4e4d-bc90-1c605e7c4f93 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.554493 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54de7fa-867c-4e4d-bc90-1c605e7c4f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.554769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54de7fa-867c-4e4d-bc90-1c605e7c4f93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.564223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54de7fa-867c-4e4d-bc90-1c605e7c4f93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:15.568228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54de7fa-867c-4e4d-bc90-1c605e7c4f93 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:15.588188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:15.588865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.589181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.589455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b54de7fa-867c-4e4d-bc90-1c605e7c4f93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:15.589636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:15.590664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:15.591549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:15.591549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:15.591549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:15.591549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:15.591549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:15.591549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:15.591549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:15.591549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:15.591989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:15.591989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:15.592141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 476/944] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:15.596863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: WARNING keystone.server.flask.common [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 11 20:45:15.599071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54de7fa-867c-4e4d-bc90-1c605e7c4f93 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:15.617627 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.sql.core [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 20:45:15.617627 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 11 20:45:15.617627 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [parameters: {'id': '508556e87a34455ab60ff9d696eb6762', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 11 20:45:15.617627 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80310) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 508556e87a34455ab60ff9d696eb6762.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 508556e87a34455ab60ff9d696eb6762. Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application conn.query(q) Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 20:45:15.632276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result.read() Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 11 20:45:15.633150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application next(self.gen) Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application yield resource Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application next(self.gen) Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise self.value Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application session.commit() Jul 11 20:45:15.634109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.session.flush() Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application compat.raise_( Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 20:45:15.635057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 20:45:15.635941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application conn.query(q) Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result.read() Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 20:45:15.636788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [parameters: {'id': '508556e87a34455ab60ff9d696eb6762', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:15.637639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:15.638414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:15.639340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:15.639340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 11 20:45:15.639340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 11 20:45:15.639340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:15.639340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:15.639340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 11 20:45:15.639340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 11 20:45:15.639340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 508556e87a34455ab60ff9d696eb6762. Jul 11 20:45:15.639340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:15.642228 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 469/945] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Jul 11 20:45:15 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:15.648152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:15.734531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54de7fa-867c-4e4d-bc90-1c605e7c4f93 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:15.776839 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981cd92-f7d4-41e4-b0c4-0c5f4b5e2e48 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:15.779042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec300c06-84d8-4a15-95bc-f0ea7c705856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:15.780865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec300c06-84d8-4a15-95bc-f0ea7c705856 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.781147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec300c06-84d8-4a15-95bc-f0ea7c705856 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.781422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec300c06-84d8-4a15-95bc-f0ea7c705856 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.782217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec300c06-84d8-4a15-95bc-f0ea7c705856 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:15.782528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec300c06-84d8-4a15-95bc-f0ea7c705856 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:15.783977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec300c06-84d8-4a15-95bc-f0ea7c705856 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:15.806367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 477/946] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:45:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:15.807961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec300c06-84d8-4a15-95bc-f0ea7c705856 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:15.834381 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:15.838161 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91f0a7-a46b-4da6-8c11-3421abdf3e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:15.838999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91f0a7-a46b-4da6-8c11-3421abdf3e01 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.839347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91f0a7-a46b-4da6-8c11-3421abdf3e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.839714 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91f0a7-a46b-4da6-8c11-3421abdf3e01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.848654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91f0a7-a46b-4da6-8c11-3421abdf3e01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:15.851035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91f0a7-a46b-4da6-8c11-3421abdf3e01 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-5f91f0a7-a46b-4da6-8c11-3421abdf3e01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:15.871587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:15.872565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:15.873421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:15.873421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:15.873421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:15.873421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:15.873421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:15.873421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:15.873421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:15.873421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:15.876171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 470/947] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:15.878683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91f0a7-a46b-4da6-8c11-3421abdf3e01 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:15.890644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:15.943972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec300c06-84d8-4a15-95bc-f0ea7c705856 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:15.985529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3aafc-7779-442f-8ee9-903ad2a649c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:15.987061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3aafc-7779-442f-8ee9-903ad2a649c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:15.987711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3aafc-7779-442f-8ee9-903ad2a649c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:15.988227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3aafc-7779-442f-8ee9-903ad2a649c1 None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/812f2c4a42fd470c9c57bd14310f320f/roles/f37ae457d604457bbe6e5dd28ce225e2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:15.991188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3aafc-7779-442f-8ee9-903ad2a649c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=812f2c4a42fd470c9c57bd14310f320f, role_id=f37ae457d604457bbe6e5dd28ce225e2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:15.993904 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3aafc-7779-442f-8ee9-903ad2a649c1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:16.006498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91f0a7-a46b-4da6-8c11-3421abdf3e01 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:16.020804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 478/948] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:45:15 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/812f2c4a42fd470c9c57bd14310f320f/roles/f37ae457d604457bbe6e5dd28ce225e2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:45:16.052874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab368c-39dd-49cc-b8ff-b6e5ac7351b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:16.053632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab368c-39dd-49cc-b8ff-b6e5ac7351b4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:16.053632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab368c-39dd-49cc-b8ff-b6e5ac7351b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:16.053916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab368c-39dd-49cc-b8ff-b6e5ac7351b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:16.054685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab368c-39dd-49cc-b8ff-b6e5ac7351b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:16.054996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab368c-39dd-49cc-b8ff-b6e5ac7351b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:16.056542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab368c-39dd-49cc-b8ff-b6e5ac7351b4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:16.079458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 471/949] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:45:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:16.923065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9760c44-d5b7-43a7-a725-802264184fea None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:16.923617 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9760c44-d5b7-43a7-a725-802264184fea None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:16.923712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9760c44-d5b7-43a7-a725-802264184fea None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:16.925349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 479/950] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:16.930835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9cc59-e2b8-4811-81a7-4cff176d617b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:16.931252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9cc59-e2b8-4811-81a7-4cff176d617b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:16.931466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9cc59-e2b8-4811-81a7-4cff176d617b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:16.949253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b6a9cc59-e2b8-4811-81a7-4cff176d617b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:16.950757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b6a9cc59-e2b8-4811-81a7-4cff176d617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:17.012950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:17.021297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 472/951] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:17.043154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9760c44-d5b7-43a7-a725-802264184fea None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:17.114719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:17.147607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9760c44-d5b7-43a7-a725-802264184fea None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:17.181325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fb107-051a-4281-b0fa-1a2e570cabc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:17.182157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fb107-051a-4281-b0fa-1a2e570cabc4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:17.182438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fb107-051a-4281-b0fa-1a2e570cabc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:17.182711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fb107-051a-4281-b0fa-1a2e570cabc4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:17.183372 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fb107-051a-4281-b0fa-1a2e570cabc4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:17.184820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fb107-051a-4281-b0fa-1a2e570cabc4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-3b3fb107-051a-4281-b0fa-1a2e570cabc4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:17.198331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:17.199391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:17.200284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 480/952] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:17.205870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9cc59-e2b8-4811-81a7-4cff176d617b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:17.309621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9cc59-e2b8-4811-81a7-4cff176d617b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:17.352959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0498ec6-d20d-4841-9522-ecd4c5e0e107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:17.353633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0498ec6-d20d-4841-9522-ecd4c5e0e107 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:17.353912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0498ec6-d20d-4841-9522-ecd4c5e0e107 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:17.354185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0498ec6-d20d-4841-9522-ecd4c5e0e107 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:17.354907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0498ec6-d20d-4841-9522-ecd4c5e0e107 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:17.355206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0498ec6-d20d-4841-9522-ecd4c5e0e107 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:17.356614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0498ec6-d20d-4841-9522-ecd4c5e0e107 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:17.361911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 473/953] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:45:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:17.367485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fb107-051a-4281-b0fa-1a2e570cabc4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:17.457292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e67da-86df-4f66-8f46-7d01a869f42e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:17.457726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e67da-86df-4f66-8f46-7d01a869f42e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:17.458240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e67da-86df-4f66-8f46-7d01a869f42e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:17.459885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 474/954] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:17.464946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ee1a0-f4e7-4434-89a4-e9198e0a03dc None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:17.465241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ee1a0-f4e7-4434-89a4-e9198e0a03dc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:17.465517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ee1a0-f4e7-4434-89a4-e9198e0a03dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:17.470455 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fb107-051a-4281-b0fa-1a2e570cabc4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:17.482866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1a0ee1a0-f4e7-4434-89a4-e9198e0a03dc None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:17.484055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1a0ee1a0-f4e7-4434-89a4-e9198e0a03dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:17.504830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aa322-67c5-4f17-8677-0eac160fb99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:17.505504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aa322-67c5-4f17-8677-0eac160fb99a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:17.505788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aa322-67c5-4f17-8677-0eac160fb99a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:17.506061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aa322-67c5-4f17-8677-0eac160fb99a None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:17.506767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321aa322-67c5-4f17-8677-0eac160fb99a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:17.507074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321aa322-67c5-4f17-8677-0eac160fb99a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:17.508539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321aa322-67c5-4f17-8677-0eac160fb99a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:17.510819 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:17.514086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 481/955] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:45:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:17.561435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 475/956] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:17.562485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:17.586961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aa322-67c5-4f17-8677-0eac160fb99a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:17.705425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aa322-67c5-4f17-8677-0eac160fb99a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:17.741987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dffee0-b74a-40af-a80c-f97b25c349e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:17.742820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dffee0-b74a-40af-a80c-f97b25c349e3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:17.743098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dffee0-b74a-40af-a80c-f97b25c349e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:17.743386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dffee0-b74a-40af-a80c-f97b25c349e3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:17.744023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dffee0-b74a-40af-a80c-f97b25c349e3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:17.745435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dffee0-b74a-40af-a80c-f97b25c349e3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f8dffee0-b74a-40af-a80c-f97b25c349e3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:17.758126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:17.759358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:17.760262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 482/957] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:17.767694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ee1a0-f4e7-4434-89a4-e9198e0a03dc None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:17.874465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ee1a0-f4e7-4434-89a4-e9198e0a03dc None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:17.909435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d0469-bc85-429c-9c5e-aade336257b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:17.910111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d0469-bc85-429c-9c5e-aade336257b1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:17.910498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d0469-bc85-429c-9c5e-aade336257b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:17.910778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d0469-bc85-429c-9c5e-aade336257b1 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:17.911585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d0469-bc85-429c-9c5e-aade336257b1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:17.911891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d0469-bc85-429c-9c5e-aade336257b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:17.913349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d0469-bc85-429c-9c5e-aade336257b1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:17.919002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 476/958] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jul 11 20:45:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:17.924528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dffee0-b74a-40af-a80c-f97b25c349e3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:17.991735 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:18.034830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dffee0-b74a-40af-a80c-f97b25c349e3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:18.072465 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14f6ca-d5cc-40b7-bd32-34dff704e5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:18.073509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14f6ca-d5cc-40b7-bd32-34dff704e5db None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.073791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14f6ca-d5cc-40b7-bd32-34dff704e5db None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.074065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14f6ca-d5cc-40b7-bd32-34dff704e5db None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.074784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14f6ca-d5cc-40b7-bd32-34dff704e5db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:18.075091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14f6ca-d5cc-40b7-bd32-34dff704e5db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:18.076492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14f6ca-d5cc-40b7-bd32-34dff704e5db None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:18.081994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 483/959] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jul 11 20:45:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:18.087688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d0469-bc85-429c-9c5e-aade336257b1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:18.132650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d8ace-fafc-49aa-a97c-7fda4822d7f6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.134114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d8ace-fafc-49aa-a97c-7fda4822d7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.134114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d8ace-fafc-49aa-a97c-7fda4822d7f6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.134929 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 484/960] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:18.140772 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39615f34-8dde-42a6-afec-3182e07aef5b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.141321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39615f34-8dde-42a6-afec-3182e07aef5b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.141819 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39615f34-8dde-42a6-afec-3182e07aef5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.148860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:18.159332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-39615f34-8dde-42a6-afec-3182e07aef5b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:18.160932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-39615f34-8dde-42a6-afec-3182e07aef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:18.195024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d0469-bc85-429c-9c5e-aade336257b1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:18.231137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f77056-bc07-4d29-b43c-a227baa511af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:18.231807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f77056-bc07-4d29-b43c-a227baa511af None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.232086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f77056-bc07-4d29-b43c-a227baa511af None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.232388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f77056-bc07-4d29-b43c-a227baa511af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.233147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f77056-bc07-4d29-b43c-a227baa511af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:18.234521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f77056-bc07-4d29-b43c-a227baa511af None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:18.240555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 485/961] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:18.256267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 477/962] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jul 11 20:45:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:18.261541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39615f34-8dde-42a6-afec-3182e07aef5b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:18.293754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb9f46-a8fe-4f13-a8cf-b9cfa274d7f7 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.294168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb9f46-a8fe-4f13-a8cf-b9cfa274d7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.294337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb9f46-a8fe-4f13-a8cf-b9cfa274d7f7 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.300229 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 478/963] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:18.301967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23514c-51de-48f9-a1ca-ea14807976e7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.302271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23514c-51de-48f9-a1ca-ea14807976e7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.302537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23514c-51de-48f9-a1ca-ea14807976e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.321316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7d23514c-51de-48f9-a1ca-ea14807976e7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:18.322968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7d23514c-51de-48f9-a1ca-ea14807976e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:18.346225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:18.404294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39615f34-8dde-42a6-afec-3182e07aef5b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:18.412569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 479/964] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:18.432973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23514c-51de-48f9-a1ca-ea14807976e7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:18.441260 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6b318-bb04-4eeb-af5b-8e245af91ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:18.442434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6b318-bb04-4eeb-af5b-8e245af91ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.442863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6b318-bb04-4eeb-af5b-8e245af91ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.443299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6b318-bb04-4eeb-af5b-8e245af91ac4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.455347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6b318-bb04-4eeb-af5b-8e245af91ac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-94e6b318-bb04-4eeb-af5b-8e245af91ac4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:18.464657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:18.465725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:18.466680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:18.468942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 486/965] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jul 11 20:45:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:18.476175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6b318-bb04-4eeb-af5b-8e245af91ac4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:18.483277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:18.586877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23514c-51de-48f9-a1ca-ea14807976e7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:18.587950 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6b318-bb04-4eeb-af5b-8e245af91ac4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:18.607071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:18.625905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6d4d5-b18f-43ec-8b00-250c931490f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:18.626588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6d4d5-b18f-43ec-8b00-250c931490f0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.626869 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6d4d5-b18f-43ec-8b00-250c931490f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.627145 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6d4d5-b18f-43ec-8b00-250c931490f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.627904 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6d4d5-b18f-43ec-8b00-250c931490f0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:18.628256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6d4d5-b18f-43ec-8b00-250c931490f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:18.630119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6d4d5-b18f-43ec-8b00-250c931490f0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:18.633544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e95c12-662e-430f-80c1-96badd2f6164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:18.634304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e95c12-662e-430f-80c1-96badd2f6164 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.634652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e95c12-662e-430f-80c1-96badd2f6164 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.634990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e95c12-662e-430f-80c1-96badd2f6164 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.636355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e95c12-662e-430f-80c1-96badd2f6164 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:18.638305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e95c12-662e-430f-80c1-96badd2f6164 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:18.639281 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 487/966] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jul 11 20:45:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:18.645711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6d4d5-b18f-43ec-8b00-250c931490f0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:18.656974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.sql.core [None req-72e95c12-662e-430f-80c1-96badd2f6164 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 20:45:18.656974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 11 20:45:18.656974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [parameters: {'user_id': '9e5f69484f2c4aecbe42ac18c9634e63', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 11 20:45:18.656974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80310) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-72e95c12-662e-430f-80c1-96badd2f6164 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application conn.query(q) Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result.read() Jul 11 20:45:18.665279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 20:45:18.666181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application next(self.gen) Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application yield resource Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application next(self.gen) Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise self.value Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application session.commit() Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 20:45:18.667142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.session.flush() Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application compat.raise_( Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 20:45:18.668050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 20:45:18.669076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application conn.query(q) Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result.read() Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 20:45:18.669922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [parameters: {'user_id': '9e5f69484f2c4aecbe42ac18c9634e63', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:18.670748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 11 20:45:18.671562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 11 20:45:18.673864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 11 20:45:18.673864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 11 20:45:18.673864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 20:45:18.673864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 20:45:18.673864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 11 20:45:18.673864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:18.673864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 480/967] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Jul 11 20:45:18 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:18.673864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e95c12-662e-430f-80c1-96badd2f6164 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:18.763477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6d4d5-b18f-43ec-8b00-250c931490f0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:18.787557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:18.791785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e95c12-662e-430f-80c1-96badd2f6164 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:18.802630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d461f-cffb-4681-9fd0-92a0c17e12fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:18.803674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d461f-cffb-4681-9fd0-92a0c17e12fa None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.804126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d461f-cffb-4681-9fd0-92a0c17e12fa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.805199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d461f-cffb-4681-9fd0-92a0c17e12fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.817523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d461f-cffb-4681-9fd0-92a0c17e12fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:18.819194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d461f-cffb-4681-9fd0-92a0c17e12fa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:18.831794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726798c-4f6a-460f-99f5-bf3fa41060a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:18.832827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726798c-4f6a-460f-99f5-bf3fa41060a4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.833109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726798c-4f6a-460f-99f5-bf3fa41060a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.833385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726798c-4f6a-460f-99f5-bf3fa41060a4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-114d461f-cffb-4681-9fd0-92a0c17e12fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:18.839996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:18.840891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:18.841769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:18.841769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:18.841769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:18.841769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:18.841994 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726798c-4f6a-460f-99f5-bf3fa41060a4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:18.847946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726798c-4f6a-460f-99f5-bf3fa41060a4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:18.848470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 488/968] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:18.849475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d461f-cffb-4681-9fd0-92a0c17e12fa None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-4726798c-4f6a-460f-99f5-bf3fa41060a4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:45:18.859803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:18.860924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:18.861855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:18.861855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:18.861855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:45:18.861855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:45:18.861855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:45:18.861855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:45:18.861855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:45:18.861855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:45:18.861855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 20:45:18.861855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:18.866154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 481/969] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:45:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:18.867199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726798c-4f6a-460f-99f5-bf3fa41060a4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:18.886898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:18.960454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d461f-cffb-4681-9fd0-92a0c17e12fa None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:18.978101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726798c-4f6a-460f-99f5-bf3fa41060a4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:18.997454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08061ea-438e-4bc7-bed2-f7153c88aa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:18.998460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08061ea-438e-4bc7-bed2-f7153c88aa08 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:18.998924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08061ea-438e-4bc7-bed2-f7153c88aa08 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:18.999375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08061ea-438e-4bc7-bed2-f7153c88aa08 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:19.000317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08061ea-438e-4bc7-bed2-f7153c88aa08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:19.000903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08061ea-438e-4bc7-bed2-f7153c88aa08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:19.002457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08061ea-438e-4bc7-bed2-f7153c88aa08 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:19.015595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ce3c1-ebf9-4015-a9bb-369e263bb5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:19.020666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce3c1-ebf9-4015-a9bb-369e263bb5ee None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:19.020666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce3c1-ebf9-4015-a9bb-369e263bb5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:19.020666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce3c1-ebf9-4015-a9bb-369e263bb5ee None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:19.020666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ce3c1-ebf9-4015-a9bb-369e263bb5ee None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:19.020666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ce3c1-ebf9-4015-a9bb-369e263bb5ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:19.020666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ce3c1-ebf9-4015-a9bb-369e263bb5ee None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:19.021499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 489/970] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:19.022647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08061ea-438e-4bc7-bed2-f7153c88aa08 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:19.072400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 482/971] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jul 11 20:45:18 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:19.123851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:19.133850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08061ea-438e-4bc7-bed2-f7153c88aa08 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:19.172677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67257f1-2cfc-4577-983c-64793eefea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:19.173814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67257f1-2cfc-4577-983c-64793eefea68 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:19.174247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67257f1-2cfc-4577-983c-64793eefea68 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:19.174666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67257f1-2cfc-4577-983c-64793eefea68 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:19.183568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67257f1-2cfc-4577-983c-64793eefea68 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:19.188177 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67257f1-2cfc-4577-983c-64793eefea68 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d67257f1-2cfc-4577-983c-64793eefea68 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:45:19.210232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:19.211270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:19.212218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:19.212218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:19.212218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:45:19.212218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:45:19.212218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:45:19.212218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:45:19.212218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:45:19.212218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:45:19.212218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 20:45:19.212218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:19.214766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 490/972] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jul 11 20:45:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:19.220411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ce3c1-ebf9-4015-a9bb-369e263bb5ee None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:19.243107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:19.335065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ce3c1-ebf9-4015-a9bb-369e263bb5ee None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:19.370907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19053f54-cb49-447d-9e9e-0bfdf219970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:19.371594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19053f54-cb49-447d-9e9e-0bfdf219970a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:19.371877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19053f54-cb49-447d-9e9e-0bfdf219970a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:19.372180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19053f54-cb49-447d-9e9e-0bfdf219970a None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:19.373038 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19053f54-cb49-447d-9e9e-0bfdf219970a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:19.373348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19053f54-cb49-447d-9e9e-0bfdf219970a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:19.374837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19053f54-cb49-447d-9e9e-0bfdf219970a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:19.392760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 483/973] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jul 11 20:45:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:19.398723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67257f1-2cfc-4577-983c-64793eefea68 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:19.506294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67257f1-2cfc-4577-983c-64793eefea68 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:19.541320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a402a46-2fb9-45dd-b841-543efcbdaa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:19.541978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a402a46-2fb9-45dd-b841-543efcbdaa77 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:19.542261 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a402a46-2fb9-45dd-b841-543efcbdaa77 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:19.542536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a402a46-2fb9-45dd-b841-543efcbdaa77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:19.564688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:19.612334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a402a46-2fb9-45dd-b841-543efcbdaa77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:19.613862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a402a46-2fb9-45dd-b841-543efcbdaa77 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:19.656001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833828c9-8b87-4339-ad9d-479c9f1f9f6e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:19.656281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833828c9-8b87-4339-ad9d-479c9f1f9f6e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:19.656502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833828c9-8b87-4339-ad9d-479c9f1f9f6e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:19.657913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 484/974] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:19.662979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ca4c6-fd34-4dc3-bae5-ec9eca51b177 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:19.663260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ca4c6-fd34-4dc3-bae5-ec9eca51b177 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:19.663519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ca4c6-fd34-4dc3-bae5-ec9eca51b177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:19.679873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e13ca4c6-fd34-4dc3-bae5-ec9eca51b177 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:19.681415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e13ca4c6-fd34-4dc3-bae5-ec9eca51b177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:19.716915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 491/975] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:19 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:45:19.723201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a402a46-2fb9-45dd-b841-543efcbdaa77 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:19.760317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 485/976] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:19.783767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ca4c6-fd34-4dc3-bae5-ec9eca51b177 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:19.785844 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:19.787926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:19.845715 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a402a46-2fb9-45dd-b841-543efcbdaa77 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:19.884371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84f798-26f6-4f79-9447-7665e5684b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:19.885183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f798-26f6-4f79-9447-7665e5684b44 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:19.885465 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f798-26f6-4f79-9447-7665e5684b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:19.885737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f798-26f6-4f79-9447-7665e5684b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:19.895742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ca4c6-fd34-4dc3-bae5-ec9eca51b177 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:19.896753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84f798-26f6-4f79-9447-7665e5684b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:19.898248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84f798-26f6-4f79-9447-7665e5684b44 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ef84f798-26f6-4f79-9447-7665e5684b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:19.916396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:19.917283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:19.918172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:19.918172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:19.918172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:19.918172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:19.918172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 492/977] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:19.923180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84f798-26f6-4f79-9447-7665e5684b44 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:19.933201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b294c8-5e09-42ef-8d77-106441366b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:19.934234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b294c8-5e09-42ef-8d77-106441366b9d None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:19.934523 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b294c8-5e09-42ef-8d77-106441366b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:19.934794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b294c8-5e09-42ef-8d77-106441366b9d None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:19.935909 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b294c8-5e09-42ef-8d77-106441366b9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:19.937324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b294c8-5e09-42ef-8d77-106441366b9d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:19.954127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 486/978] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:45:19 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:45:20.036675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84f798-26f6-4f79-9447-7665e5684b44 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:20.039390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:20.085532 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3c51e-b2bf-4081-9358-2db49c419be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:20.086218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3c51e-b2bf-4081-9358-2db49c419be5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:20.086501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3c51e-b2bf-4081-9358-2db49c419be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:20.086777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3c51e-b2bf-4081-9358-2db49c419be5 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:20.087513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3c51e-b2bf-4081-9358-2db49c419be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:20.087817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3c51e-b2bf-4081-9358-2db49c419be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:20.089270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3c51e-b2bf-4081-9358-2db49c419be5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:20.102344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 493/979] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:20.108146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b294c8-5e09-42ef-8d77-106441366b9d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:20.178248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:20.218935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b294c8-5e09-42ef-8d77-106441366b9d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:20.253070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b20796-1be0-41eb-ba3c-c83bb52296fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:20.253980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20796-1be0-41eb-ba3c-c83bb52296fc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:20.254335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20796-1be0-41eb-ba3c-c83bb52296fc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:20.254615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20796-1be0-41eb-ba3c-c83bb52296fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:20.261589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b20796-1be0-41eb-ba3c-c83bb52296fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:20.263032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b20796-1be0-41eb-ba3c-c83bb52296fc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f6b20796-1be0-41eb-ba3c-c83bb52296fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:20.277588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:20.278459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:20.279342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:20.279342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:20.279342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:20.279342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:20.279342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:20.279342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:20.279342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:20.279342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:20.279342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 487/980] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:20.284951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3c51e-b2bf-4081-9358-2db49c419be5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:20.382134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726dca0-f8c9-496e-9915-c141f8dea8e7 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:20.382433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726dca0-f8c9-496e-9915-c141f8dea8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:20.382690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726dca0-f8c9-496e-9915-c141f8dea8e7 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:20.384096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 488/981] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:20.389058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67eaab-5980-4bae-9886-bccf3055dab3 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:20.389362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67eaab-5980-4bae-9886-bccf3055dab3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:20.389605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67eaab-5980-4bae-9886-bccf3055dab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:20.396232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3c51e-b2bf-4081-9358-2db49c419be5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:20.407058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8f67eaab-5980-4bae-9886-bccf3055dab3 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:20.408292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8f67eaab-5980-4bae-9886-bccf3055dab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:20.430923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226225b8-8896-4fbe-81b7-f5069f253d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:20.431514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226225b8-8896-4fbe-81b7-f5069f253d4c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:20.431790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226225b8-8896-4fbe-81b7-f5069f253d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:20.432056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226225b8-8896-4fbe-81b7-f5069f253d4c None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:20.432886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226225b8-8896-4fbe-81b7-f5069f253d4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:20.433191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226225b8-8896-4fbe-81b7-f5069f253d4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:20.434621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226225b8-8896-4fbe-81b7-f5069f253d4c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:20.447305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 494/982] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:20.453646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226225b8-8896-4fbe-81b7-f5069f253d4c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:20.477873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:20.489023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 489/983] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:20.511608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67eaab-5980-4bae-9886-bccf3055dab3 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:20.564232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226225b8-8896-4fbe-81b7-f5069f253d4c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:20.569934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:20.601606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7b9d5-177c-4010-b509-0b11722aeda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:20.602255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7b9d5-177c-4010-b509-0b11722aeda0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:20.602554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7b9d5-177c-4010-b509-0b11722aeda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:20.602829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7b9d5-177c-4010-b509-0b11722aeda0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:20.603664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7b9d5-177c-4010-b509-0b11722aeda0 None admin] RBAC: Adding query filter params (role.id=f37ae457d604457bbe6e5dd28ce225e2, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=812f2c4a42fd470c9c57bd14310f320f) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:20.603974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7b9d5-177c-4010-b509-0b11722aeda0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:20.605450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7b9d5-177c-4010-b509-0b11722aeda0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:20.620155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 495/984] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:20 2022] GET /identity/v3/role_assignments?user.id=812f2c4a42fd470c9c57bd14310f320f&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=f37ae457d604457bbe6e5dd28ce225e2 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:20.629301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67eaab-5980-4bae-9886-bccf3055dab3 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:20.667179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6eacc-c0b7-4fa2-9f3e-1a0447711893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:20.668025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6eacc-c0b7-4fa2-9f3e-1a0447711893 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:20.668336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6eacc-c0b7-4fa2-9f3e-1a0447711893 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:20.668608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6eacc-c0b7-4fa2-9f3e-1a0447711893 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:20.669317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6eacc-c0b7-4fa2-9f3e-1a0447711893 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:20.670736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6eacc-c0b7-4fa2-9f3e-1a0447711893 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-90f6eacc-c0b7-4fa2-9f3e-1a0447711893 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:20.683993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:20.685251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:20.691896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 490/985] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:20.695142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7b9d5-177c-4010-b509-0b11722aeda0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:20.792576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:20.809625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7b9d5-177c-4010-b509-0b11722aeda0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:20.850070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7f286-255f-43a6-95fa-cab0ccf60616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:20.850791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7f286-255f-43a6-95fa-cab0ccf60616 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:20.851077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7f286-255f-43a6-95fa-cab0ccf60616 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:20.851350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7f286-255f-43a6-95fa-cab0ccf60616 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:20.852059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7f286-255f-43a6-95fa-cab0ccf60616 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:20.852465 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7f286-255f-43a6-95fa-cab0ccf60616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:20.853850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7f286-255f-43a6-95fa-cab0ccf60616 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:20.863257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 496/986] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jul 11 20:45:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:20.864517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6eacc-c0b7-4fa2-9f3e-1a0447711893 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:20.969152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6eacc-c0b7-4fa2-9f3e-1a0447711893 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:20.987067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:21.005003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfd1c4-40a3-43e9-a85b-1691d8486905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:21.005678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfd1c4-40a3-43e9-a85b-1691d8486905 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:21.005960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfd1c4-40a3-43e9-a85b-1691d8486905 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:21.006236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfd1c4-40a3-43e9-a85b-1691d8486905 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:21.006950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfd1c4-40a3-43e9-a85b-1691d8486905 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:21.007258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfd1c4-40a3-43e9-a85b-1691d8486905 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:21.008681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfd1c4-40a3-43e9-a85b-1691d8486905 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:21.014156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 491/987] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jul 11 20:45:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:21.020085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7f286-255f-43a6-95fa-cab0ccf60616 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:21.124296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7f286-255f-43a6-95fa-cab0ccf60616 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:21.160391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017681e8-1524-4ce9-b28c-49815b52b1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:21.161172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017681e8-1524-4ce9-b28c-49815b52b1da None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:21.161453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017681e8-1524-4ce9-b28c-49815b52b1da None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:21.161724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017681e8-1524-4ce9-b28c-49815b52b1da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:21.162534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017681e8-1524-4ce9-b28c-49815b52b1da None admin] RBAC: Adding query filter params (interface=admin, service_id=b3026f959b5149e8b3919365729883fa, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:21.162839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017681e8-1524-4ce9-b28c-49815b52b1da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:21.164228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017681e8-1524-4ce9-b28c-49815b52b1da None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:21.169532 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 497/988] 10.222.0.20 () {56 vars in 1249 bytes} [Mon Jul 11 20:45:21 2022] GET /identity/v3/endpoints?service_id=b3026f959b5149e8b3919365729883fa&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:21.175226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfd1c4-40a3-43e9-a85b-1691d8486905 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:21.279615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfd1c4-40a3-43e9-a85b-1691d8486905 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:21.313074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537427b6-31f0-4c1f-a8a3-9cbe6c0edb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:21.313743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537427b6-31f0-4c1f-a8a3-9cbe6c0edb08 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:21.314021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537427b6-31f0-4c1f-a8a3-9cbe6c0edb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:21.314292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537427b6-31f0-4c1f-a8a3-9cbe6c0edb08 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:21.314979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537427b6-31f0-4c1f-a8a3-9cbe6c0edb08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:21.316414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537427b6-31f0-4c1f-a8a3-9cbe6c0edb08 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:21.323263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 492/989] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jul 11 20:45:21 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:45:22.019019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad1368-3e04-452f-82c2-51f14e31259d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:22.019428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad1368-3e04-452f-82c2-51f14e31259d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:22.019666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad1368-3e04-452f-82c2-51f14e31259d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:22.021076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 498/990] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:22.026476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860d6f-bbed-4792-b1a2-36ce2e553f58 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:22.026769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860d6f-bbed-4792-b1a2-36ce2e553f58 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:22.027006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860d6f-bbed-4792-b1a2-36ce2e553f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:22.044930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-18860d6f-bbed-4792-b1a2-36ce2e553f58 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:22.046100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-18860d6f-bbed-4792-b1a2-36ce2e553f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:22.090846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:22.119278 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 493/991] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:22.141439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad1368-3e04-452f-82c2-51f14e31259d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:22.156790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:22.244886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad1368-3e04-452f-82c2-51f14e31259d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:22.278816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c63536-585a-4b57-b7a1-af3f4ce3a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:22.279614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c63536-585a-4b57-b7a1-af3f4ce3a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:22.279892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c63536-585a-4b57-b7a1-af3f4ce3a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:22.280152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c63536-585a-4b57-b7a1-af3f4ce3a3e2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:22.280834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c63536-585a-4b57-b7a1-af3f4ce3a3e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:22.282195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c63536-585a-4b57-b7a1-af3f4ce3a3e2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a4c63536-585a-4b57-b7a1-af3f4ce3a3e2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:22.294052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:22.295056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 20:45:22.295888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:22.297267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 499/992] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:22.301415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18860d6f-bbed-4792-b1a2-36ce2e553f58 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:22.407842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18860d6f-bbed-4792-b1a2-36ce2e553f58 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:22.445885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd31fa-e860-4e17-979f-bd3f40ae7c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:22.446554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd31fa-e860-4e17-979f-bd3f40ae7c10 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:22.446828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd31fa-e860-4e17-979f-bd3f40ae7c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:22.447092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd31fa-e860-4e17-979f-bd3f40ae7c10 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:22.448184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd31fa-e860-4e17-979f-bd3f40ae7c10 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:22.448487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd31fa-e860-4e17-979f-bd3f40ae7c10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:22.449861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd31fa-e860-4e17-979f-bd3f40ae7c10 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:22.455039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 494/993] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:45:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:22.460900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c63536-585a-4b57-b7a1-af3f4ce3a3e2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:22.539755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:22.563742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c63536-585a-4b57-b7a1-af3f4ce3a3e2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:22.597208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3174544-a59e-47ff-b286-65cd8b39fb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:22.597964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3174544-a59e-47ff-b286-65cd8b39fb5b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:22.598304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3174544-a59e-47ff-b286-65cd8b39fb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:22.598599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3174544-a59e-47ff-b286-65cd8b39fb5b None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:22.599302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3174544-a59e-47ff-b286-65cd8b39fb5b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:22.599622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3174544-a59e-47ff-b286-65cd8b39fb5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:22.601001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3174544-a59e-47ff-b286-65cd8b39fb5b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:22.605947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 500/994] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:45:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:22.611605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd31fa-e860-4e17-979f-bd3f40ae7c10 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:22.654805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:22.677469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc715d-1dce-4d9f-95a8-fb7f5ad6bd34 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:22.677729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc715d-1dce-4d9f-95a8-fb7f5ad6bd34 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:22.677952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc715d-1dce-4d9f-95a8-fb7f5ad6bd34 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:22.679326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 501/995] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:22.684923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba455317-7fa4-4ab2-ad7a-3514e8d7e2d7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:22.685190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba455317-7fa4-4ab2-ad7a-3514e8d7e2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:22.685472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba455317-7fa4-4ab2-ad7a-3514e8d7e2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:22.703324 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ba455317-7fa4-4ab2-ad7a-3514e8d7e2d7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:22.704534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ba455317-7fa4-4ab2-ad7a-3514e8d7e2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:22.717968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd31fa-e860-4e17-979f-bd3f40ae7c10 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:22.753963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7967028-e300-45d6-a205-5f8330d5a61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:22.754909 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7967028-e300-45d6-a205-5f8330d5a61c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:22.755201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7967028-e300-45d6-a205-5f8330d5a61c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:22.755816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7967028-e300-45d6-a205-5f8330d5a61c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:22.756636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7967028-e300-45d6-a205-5f8330d5a61c None admin] RBAC: Adding query filter params (interface=public, service_id=27b6faf044ee46f8a726734211666f91, region_id=RegionOne) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:22.756964 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7967028-e300-45d6-a205-5f8330d5a61c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:22.758344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7967028-e300-45d6-a205-5f8330d5a61c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:22.764157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 495/996] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jul 11 20:45:22 2022] GET /identity/v3/endpoints?service_id=27b6faf044ee46f8a726734211666f91&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:22.771795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7967028-e300-45d6-a205-5f8330d5a61c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:22.777484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 502/997] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:22.799647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba455317-7fa4-4ab2-ad7a-3514e8d7e2d7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:22.884955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:22.909143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7967028-e300-45d6-a205-5f8330d5a61c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:22.913129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:22.917064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba455317-7fa4-4ab2-ad7a-3514e8d7e2d7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:22.951381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad3194-de3b-46d6-97c2-9ade3a01de8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:22.951697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1724432-650f-4f42-b73e-7b416389e895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:22.951969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad3194-de3b-46d6-97c2-9ade3a01de8c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:22.952277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad3194-de3b-46d6-97c2-9ade3a01de8c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:22.952508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1724432-650f-4f42-b73e-7b416389e895 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:22.952611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad3194-de3b-46d6-97c2-9ade3a01de8c None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:22.952784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1724432-650f-4f42-b73e-7b416389e895 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:22.953062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1724432-650f-4f42-b73e-7b416389e895 None admin] PATH_INFO: `/v3/services/image` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:22.953221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad3194-de3b-46d6-97c2-9ade3a01de8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:22.953725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1724432-650f-4f42-b73e-7b416389e895 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:22.954586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad3194-de3b-46d6-97c2-9ade3a01de8c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:22.955079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1724432-650f-4f42-b73e-7b416389e895 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:22.961555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 496/998] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jul 11 20:45:22 2022] GET /identity/v3/services => generated 1727 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d1724432-650f-4f42-b73e-7b416389e895 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:22.966912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:22.968022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 20:45:22.968905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:22.972519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 503/999] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Jul 11 20:45:22 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:22.974414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad3194-de3b-46d6-97c2-9ade3a01de8c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:23.088312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad3194-de3b-46d6-97c2-9ade3a01de8c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:23.128187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb361d-834b-408e-8318-d379545e8cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:23.128559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb361d-834b-408e-8318-d379545e8cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:23.128726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb361d-834b-408e-8318-d379545e8cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:23.129129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb361d-834b-408e-8318-d379545e8cf5 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:23.129924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb361d-834b-408e-8318-d379545e8cf5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:23.130235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb361d-834b-408e-8318-d379545e8cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:23.131640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb361d-834b-408e-8318-d379545e8cf5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:23.138012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 497/1000] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:23.143653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1724432-650f-4f42-b73e-7b416389e895 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:23.255090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1724432-650f-4f42-b73e-7b416389e895 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:23.289659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14cfca-3396-4545-9fe8-cb5425131897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:23.290335 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14cfca-3396-4545-9fe8-cb5425131897 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:23.290618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14cfca-3396-4545-9fe8-cb5425131897 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:23.290887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14cfca-3396-4545-9fe8-cb5425131897 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:23.291594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14cfca-3396-4545-9fe8-cb5425131897 None admin] RBAC: Adding query filter params (type=image) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:23.291884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14cfca-3396-4545-9fe8-cb5425131897 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:23.293311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14cfca-3396-4545-9fe8-cb5425131897 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:23.294802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:23.299592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 504/1001] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:23 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:23.343786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f1b81-5247-4a10-ac1a-b9ff794c8858 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:23.344402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f1b81-5247-4a10-ac1a-b9ff794c8858 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:23.344402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f1b81-5247-4a10-ac1a-b9ff794c8858 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:23.346006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 498/1002] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:23.352048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba35a53-046f-45f2-b01e-530c958dc3fd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:23.352416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba35a53-046f-45f2-b01e-530c958dc3fd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:23.352601 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba35a53-046f-45f2-b01e-530c958dc3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:23.371052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-dba35a53-046f-45f2-b01e-530c958dc3fd None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:23.372316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dba35a53-046f-45f2-b01e-530c958dc3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:23.451077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 505/1003] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:23.474438 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f1b81-5247-4a10-ac1a-b9ff794c8858 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:23.510111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:23.585896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f1b81-5247-4a10-ac1a-b9ff794c8858 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:23.620120 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3c1b3-f637-4482-934a-a86845a68780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:23.620968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3c1b3-f637-4482-934a-a86845a68780 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:23.621250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3c1b3-f637-4482-934a-a86845a68780 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:23.621525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3c1b3-f637-4482-934a-a86845a68780 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:23.622190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3c1b3-f637-4482-934a-a86845a68780 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:23.623568 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3c1b3-f637-4482-934a-a86845a68780 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-58b3c1b3-f637-4482-934a-a86845a68780 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:23.635848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:23.636880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 20:45:23.637700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:23.639035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 499/1004] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jul 11 20:45:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:23.643415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba35a53-046f-45f2-b01e-530c958dc3fd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:23.748534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba35a53-046f-45f2-b01e-530c958dc3fd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:23.775961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:23.785507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023886d-7ef0-4503-882c-87a2c96d9954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:23.786331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023886d-7ef0-4503-882c-87a2c96d9954 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:23.786615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023886d-7ef0-4503-882c-87a2c96d9954 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:23.786888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023886d-7ef0-4503-882c-87a2c96d9954 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:23.787599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023886d-7ef0-4503-882c-87a2c96d9954 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:23.787903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023886d-7ef0-4503-882c-87a2c96d9954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:23.789323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023886d-7ef0-4503-882c-87a2c96d9954 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:23.798941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 506/1005] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jul 11 20:45:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:23.799942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3c1b3-f637-4482-934a-a86845a68780 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:23.903855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:23.911182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3c1b3-f637-4482-934a-a86845a68780 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:23.945264 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c6ee2-ccdd-4d0b-a905-5c51942a5150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:23.945950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c6ee2-ccdd-4d0b-a905-5c51942a5150 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:23.946231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c6ee2-ccdd-4d0b-a905-5c51942a5150 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:23.946503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c6ee2-ccdd-4d0b-a905-5c51942a5150 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:23.947220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c6ee2-ccdd-4d0b-a905-5c51942a5150 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:23.947895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c6ee2-ccdd-4d0b-a905-5c51942a5150 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:23.949319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c6ee2-ccdd-4d0b-a905-5c51942a5150 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:23.958964 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 500/1006] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jul 11 20:45:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:23.960148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023886d-7ef0-4503-882c-87a2c96d9954 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:24.062194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023886d-7ef0-4503-882c-87a2c96d9954 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:24.097396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25eff0-d746-4050-9920-47a314722949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:24.098349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25eff0-d746-4050-9920-47a314722949 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:24.098814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25eff0-d746-4050-9920-47a314722949 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:24.099098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25eff0-d746-4050-9920-47a314722949 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:24.099896 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25eff0-d746-4050-9920-47a314722949 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:24.101328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25eff0-d746-4050-9920-47a314722949 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:24.128131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 507/1007] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jul 11 20:45:23 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:25.020713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82e05c-d6f2-443a-9132-713dc67a2617 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:25.021355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82e05c-d6f2-443a-9132-713dc67a2617 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:25.021355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82e05c-d6f2-443a-9132-713dc67a2617 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:25.022756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 501/1008] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:25.028096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0604c4-6144-4012-aa5e-2f0cac180be7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:25.028558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0604c4-6144-4012-aa5e-2f0cac180be7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:25.028729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0604c4-6144-4012-aa5e-2f0cac180be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:25.046816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5e0604c4-6144-4012-aa5e-2f0cac180be7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:25.047988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5e0604c4-6144-4012-aa5e-2f0cac180be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:25.097515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:25.128283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 508/1009] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:25 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:25.149817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82e05c-d6f2-443a-9132-713dc67a2617 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:25.265415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82e05c-d6f2-443a-9132-713dc67a2617 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:25.301046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f595ef-4df5-4bbc-9a50-eb61eda7f84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:25.301854 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f595ef-4df5-4bbc-9a50-eb61eda7f84c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:25.302134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f595ef-4df5-4bbc-9a50-eb61eda7f84c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:25.302410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f595ef-4df5-4bbc-9a50-eb61eda7f84c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:25.303066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f595ef-4df5-4bbc-9a50-eb61eda7f84c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:25.304482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f595ef-4df5-4bbc-9a50-eb61eda7f84c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-16f595ef-4df5-4bbc-9a50-eb61eda7f84c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:25.316411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:25.317398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:25.323971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 502/1010] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:25.324993 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0604c4-6144-4012-aa5e-2f0cac180be7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:25.347830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0052-03b7-4bfd-aca5-520214aca1f8 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:25.348141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0052-03b7-4bfd-aca5-520214aca1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:25.348342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0052-03b7-4bfd-aca5-520214aca1f8 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:25.349976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 503/1011] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:25.354876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc115c80-3c40-4d08-bb26-6eff0ca22fa8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:25.355168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc115c80-3c40-4d08-bb26-6eff0ca22fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:25.355433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc115c80-3c40-4d08-bb26-6eff0ca22fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:25.374307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bc115c80-3c40-4d08-bb26-6eff0ca22fa8 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:25.375485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bc115c80-3c40-4d08-bb26-6eff0ca22fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:25.393268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:25.435810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0604c4-6144-4012-aa5e-2f0cac180be7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:25.445128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 504/1012] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:25 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:25.467377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc115c80-3c40-4d08-bb26-6eff0ca22fa8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:25.470946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d605c4c-0207-4177-82b5-40ac4d024793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:25.471161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605c4c-0207-4177-82b5-40ac4d024793 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:25.471402 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605c4c-0207-4177-82b5-40ac4d024793 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:25.471672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605c4c-0207-4177-82b5-40ac4d024793 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:25.472408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d605c4c-0207-4177-82b5-40ac4d024793 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:25.472847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d605c4c-0207-4177-82b5-40ac4d024793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:25.474182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d605c4c-0207-4177-82b5-40ac4d024793 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:25.483893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 509/1013] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:45:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:25.484988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d605c4c-0207-4177-82b5-40ac4d024793 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:25.531643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:25.595716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc115c80-3c40-4d08-bb26-6eff0ca22fa8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:25.621240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d605c4c-0207-4177-82b5-40ac4d024793 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:25.646901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d963366-6287-4afd-b8a6-2426295171d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:25.647578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d963366-6287-4afd-b8a6-2426295171d4 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:25.647858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d963366-6287-4afd-b8a6-2426295171d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:25.648172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d963366-6287-4afd-b8a6-2426295171d4 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:25.649168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d963366-6287-4afd-b8a6-2426295171d4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:25.650545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d963366-6287-4afd-b8a6-2426295171d4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:25.679652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 505/1014] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:45:25 2022] POST /identity/v3/services => generated 242 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:45:25.692946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667378aa-078e-4c58-93b4-982f8cb3fd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:25.693970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667378aa-078e-4c58-93b4-982f8cb3fd91 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:25.694406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667378aa-078e-4c58-93b4-982f8cb3fd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:25.694838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667378aa-078e-4c58-93b4-982f8cb3fd91 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:25.695898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667378aa-078e-4c58-93b4-982f8cb3fd91 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:25.696559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667378aa-078e-4c58-93b4-982f8cb3fd91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:25.704167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667378aa-078e-4c58-93b4-982f8cb3fd91 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:25.710226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 510/1015] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jul 11 20:45:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:25.720647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d963366-6287-4afd-b8a6-2426295171d4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:25.741260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:25.849525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d963366-6287-4afd-b8a6-2426295171d4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:25.888077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffa630-6196-415f-bc3e-3f1b095cdbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:25.888708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffa630-6196-415f-bc3e-3f1b095cdbf9 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:25.888987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffa630-6196-415f-bc3e-3f1b095cdbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:25.889257 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffa630-6196-415f-bc3e-3f1b095cdbf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:25.890716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffa630-6196-415f-bc3e-3f1b095cdbf9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:25.891382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffa630-6196-415f-bc3e-3f1b095cdbf9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:25.912778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 506/1016] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jul 11 20:45:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:26.227580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86382f50-7e50-4b61-80b1-ea653cb7b98d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:26.228131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86382f50-7e50-4b61-80b1-ea653cb7b98d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:26.228131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86382f50-7e50-4b61-80b1-ea653cb7b98d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:26.229428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 511/1017] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:26.235174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8036-e782-4042-a603-163e7177d505 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:26.235632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8036-e782-4042-a603-163e7177d505 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:26.235809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8036-e782-4042-a603-163e7177d505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:26.253382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-700b8036-e782-4042-a603-163e7177d505 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:26.254946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-700b8036-e782-4042-a603-163e7177d505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:26.320137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:26.333379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 507/1018] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:26.356694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86382f50-7e50-4b61-80b1-ea653cb7b98d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:26.462474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86382f50-7e50-4b61-80b1-ea653cb7b98d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:26.464052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:26.500565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcd73f-eda8-4a50-a723-104b78556439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:26.501282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcd73f-eda8-4a50-a723-104b78556439 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:26.501558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcd73f-eda8-4a50-a723-104b78556439 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:26.501834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcd73f-eda8-4a50-a723-104b78556439 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:26.503076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcd73f-eda8-4a50-a723-104b78556439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:26.504673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcd73f-eda8-4a50-a723-104b78556439 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:26.506516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 512/1019] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:26.514375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b8036-e782-4042-a603-163e7177d505 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:26.615280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b8036-e782-4042-a603-163e7177d505 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:26.649167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a146e-6fc7-4b02-b548-7ea1b7b28027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:26.649829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a146e-6fc7-4b02-b548-7ea1b7b28027 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:26.650109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a146e-6fc7-4b02-b548-7ea1b7b28027 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:26.650380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a146e-6fc7-4b02-b548-7ea1b7b28027 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:26.651087 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a146e-6fc7-4b02-b548-7ea1b7b28027 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:26.652579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a146e-6fc7-4b02-b548-7ea1b7b28027 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:26.678458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 508/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:45:26 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:27.757941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcb1d4-d373-4690-af11-35ab3f3c2c75 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:27.758484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcb1d4-d373-4690-af11-35ab3f3c2c75 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:27.758484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcb1d4-d373-4690-af11-35ab3f3c2c75 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:27.759845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 513/1021] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:27.765414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8ba75-596a-4355-ad7b-fab0f45bb93b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:27.765815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8ba75-596a-4355-ad7b-fab0f45bb93b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:27.765960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8ba75-596a-4355-ad7b-fab0f45bb93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:27.783674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-fef8ba75-596a-4355-ad7b-fab0f45bb93b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:27.784990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fef8ba75-596a-4355-ad7b-fab0f45bb93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:27.848143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:27.876742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 509/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:27.899275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcb1d4-d373-4690-af11-35ab3f3c2c75 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:28.012149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcb1d4-d373-4690-af11-35ab3f3c2c75 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:28.049813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c132ba6-3086-4180-b44f-aa19ce0c3a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:28.050936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132ba6-3086-4180-b44f-aa19ce0c3a9d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.051232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132ba6-3086-4180-b44f-aa19ce0c3a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.051509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132ba6-3086-4180-b44f-aa19ce0c3a9d None admin] PATH_INFO: `/v3/services/image` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.052192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c132ba6-3086-4180-b44f-aa19ce0c3a9d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:28.053572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c132ba6-3086-4180-b44f-aa19ce0c3a9d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-2c132ba6-3086-4180-b44f-aa19ce0c3a9d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:28.065110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:28.066051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:28.072665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 514/1023] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Jul 11 20:45:27 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:28.073471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8ba75-596a-4355-ad7b-fab0f45bb93b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:28.117201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae491552-71fe-4e98-9b24-9b5f5b9b268c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.117548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae491552-71fe-4e98-9b24-9b5f5b9b268c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.117748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae491552-71fe-4e98-9b24-9b5f5b9b268c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.119401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 515/1024] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:28.124575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17267e22-bb3e-4d77-b734-7f365f1fad1e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.124858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17267e22-bb3e-4d77-b734-7f365f1fad1e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.125133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17267e22-bb3e-4d77-b734-7f365f1fad1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.146196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-17267e22-bb3e-4d77-b734-7f365f1fad1e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:28.148057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-17267e22-bb3e-4d77-b734-7f365f1fad1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:28.179818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:28.180624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8ba75-596a-4355-ad7b-fab0f45bb93b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:28.222034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fcef4-4fea-4a95-9a44-0a736652895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:28.223436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 516/1025] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:28.224208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fcef4-4fea-4a95-9a44-0a736652895c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.224558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fcef4-4fea-4a95-9a44-0a736652895c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.224774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fcef4-4fea-4a95-9a44-0a736652895c None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.225516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fcef4-4fea-4a95-9a44-0a736652895c None admin] RBAC: Adding query filter params (name=image) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:28.225822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fcef4-4fea-4a95-9a44-0a736652895c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:28.227334 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fcef4-4fea-4a95-9a44-0a736652895c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:28.237735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 510/1026] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:28.240584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17267e22-bb3e-4d77-b734-7f365f1fad1e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:28.249140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fcef4-4fea-4a95-9a44-0a736652895c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:28.297604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:28.362458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17267e22-bb3e-4d77-b734-7f365f1fad1e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:28.363993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fcef4-4fea-4a95-9a44-0a736652895c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:28.397711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181e1a-07bb-4fa3-9459-7e0fde5d3617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:28.398377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181e1a-07bb-4fa3-9459-7e0fde5d3617 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.398654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181e1a-07bb-4fa3-9459-7e0fde5d3617 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.398923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181e1a-07bb-4fa3-9459-7e0fde5d3617 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.399675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83181e1a-07bb-4fa3-9459-7e0fde5d3617 None admin] RBAC: Adding query filter params (type=image) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:28.399959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83181e1a-07bb-4fa3-9459-7e0fde5d3617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:28.401549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83181e1a-07bb-4fa3-9459-7e0fde5d3617 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:28.412345 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 517/1027] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:28.413542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181e1a-07bb-4fa3-9459-7e0fde5d3617 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:28.413664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0adf2-5869-4490-b443-f70942682b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:28.414452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0adf2-5869-4490-b443-f70942682b95 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.414856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0adf2-5869-4490-b443-f70942682b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.415135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0adf2-5869-4490-b443-f70942682b95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.431846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0adf2-5869-4490-b443-f70942682b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:28.441621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9dc0adf2-5869-4490-b443-f70942682b95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:28.444596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:28.445791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:28.452507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 511/1028] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jul 11 20:45:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:28.453206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0adf2-5869-4490-b443-f70942682b95 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:28.543458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181e1a-07bb-4fa3-9459-7e0fde5d3617 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:28.581297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0adf2-5869-4490-b443-f70942682b95 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:28.583286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:28.590955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43d2aa-6ede-4802-891b-5af0804b1ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:28.591635 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d2aa-6ede-4802-891b-5af0804b1ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.591918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d2aa-6ede-4802-891b-5af0804b1ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.592229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d2aa-6ede-4802-891b-5af0804b1ef7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.593025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43d2aa-6ede-4802-891b-5af0804b1ef7 None admin] RBAC: Adding query filter params (interface=public, service_id=b88dd656a3b84e79a34fa3a24a1e1e5a, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:28.593342 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43d2aa-6ede-4802-891b-5af0804b1ef7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:28.594725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43d2aa-6ede-4802-891b-5af0804b1ef7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:28.604155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 518/1029] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jul 11 20:45:28 2022] GET /identity/v3/endpoints?service_id=b88dd656a3b84e79a34fa3a24a1e1e5a&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:28.606467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43d2aa-6ede-4802-891b-5af0804b1ef7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:28.639976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f9193-a87c-43cb-ac90-110a042581cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:28.641190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f9193-a87c-43cb-ac90-110a042581cb None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.641594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f9193-a87c-43cb-ac90-110a042581cb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.641881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f9193-a87c-43cb-ac90-110a042581cb None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.642828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f9193-a87c-43cb-ac90-110a042581cb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:28.643463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f9193-a87c-43cb-ac90-110a042581cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:28.645716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f9193-a87c-43cb-ac90-110a042581cb None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:28.657572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 512/1030] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jul 11 20:45:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:28.663201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f9193-a87c-43cb-ac90-110a042581cb None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:28.719709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:28.729687 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43d2aa-6ede-4802-891b-5af0804b1ef7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:28.785099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e948dc2-62b6-4da0-9f1c-64866c64cdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:28.786155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e948dc2-62b6-4da0-9f1c-64866c64cdcd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.786442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e948dc2-62b6-4da0-9f1c-64866c64cdcd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.789644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e948dc2-62b6-4da0-9f1c-64866c64cdcd None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.789644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e948dc2-62b6-4da0-9f1c-64866c64cdcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:28.789644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e948dc2-62b6-4da0-9f1c-64866c64cdcd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:28.804144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 519/1031] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jul 11 20:45:28 2022] GET /identity/v3/services => generated 1957 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:45:28.808840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f9193-a87c-43cb-ac90-110a042581cb None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:28.861654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f707-16c8-45e7-a632-efd106577652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:28.862503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f707-16c8-45e7-a632-efd106577652 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.862782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f707-16c8-45e7-a632-efd106577652 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.863509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f707-16c8-45e7-a632-efd106577652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.879671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35f707-16c8-45e7-a632-efd106577652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:28.882538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35f707-16c8-45e7-a632-efd106577652 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-1e35f707-16c8-45e7-a632-efd106577652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:28.904576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:28.905562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:28.906446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:28.906446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:28.906446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:28.906446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:28.908523 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 513/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:28.912492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f707-16c8-45e7-a632-efd106577652 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:28.913434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af43c6f-6df3-4414-9c8e-51209addbfb1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.913944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af43c6f-6df3-4414-9c8e-51209addbfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.914409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af43c6f-6df3-4414-9c8e-51209addbfb1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.917774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 520/1033] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:28.927291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db586c5-8d37-48c9-85e6-3a5f121ae4ab None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:28.927658 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db586c5-8d37-48c9-85e6-3a5f121ae4ab None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:28.927998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db586c5-8d37-48c9-85e6-3a5f121ae4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:28.939360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:28.948470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9db586c5-8d37-48c9-85e6-3a5f121ae4ab None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:28.949976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9db586c5-8d37-48c9-85e6-3a5f121ae4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:29.042970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 521/1034] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:29.048014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f707-16c8-45e7-a632-efd106577652 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:29.076049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db586c5-8d37-48c9-85e6-3a5f121ae4ab None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:29.091370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad23cd4-23f3-44f3-8cf7-ba720f537c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:29.092390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad23cd4-23f3-44f3-8cf7-ba720f537c7d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:29.092859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad23cd4-23f3-44f3-8cf7-ba720f537c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:29.093328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad23cd4-23f3-44f3-8cf7-ba720f537c7d None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:29.094659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad23cd4-23f3-44f3-8cf7-ba720f537c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:29.095185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad23cd4-23f3-44f3-8cf7-ba720f537c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:29.097668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad23cd4-23f3-44f3-8cf7-ba720f537c7d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:29.116897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 514/1035] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:29.123075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad23cd4-23f3-44f3-8cf7-ba720f537c7d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:29.202541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:29.222522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db586c5-8d37-48c9-85e6-3a5f121ae4ab None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:29.227006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:29.257549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad23cd4-23f3-44f3-8cf7-ba720f537c7d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:29.273386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40d18e-7f92-4cbc-8b97-1dfee6e70c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:29.274572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40d18e-7f92-4cbc-8b97-1dfee6e70c4a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:29.274882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40d18e-7f92-4cbc-8b97-1dfee6e70c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:29.275157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40d18e-7f92-4cbc-8b97-1dfee6e70c4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:29.276402 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40d18e-7f92-4cbc-8b97-1dfee6e70c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:29.277853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40d18e-7f92-4cbc-8b97-1dfee6e70c4a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:29.279915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 522/1036] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:29.288427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40d18e-7f92-4cbc-8b97-1dfee6e70c4a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:29.300706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b540ba-18df-4ed5-9745-e537e2dcfd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:29.301661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b540ba-18df-4ed5-9745-e537e2dcfd48 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:29.301941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b540ba-18df-4ed5-9745-e537e2dcfd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:29.302216 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b540ba-18df-4ed5-9745-e537e2dcfd48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:29.311041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b540ba-18df-4ed5-9745-e537e2dcfd48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:29.312808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b540ba-18df-4ed5-9745-e537e2dcfd48 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a9b540ba-18df-4ed5-9745-e537e2dcfd48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:45:29.334413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:29.335434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:29.336401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:29.336401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:29.336401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:45:29.336401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:45:29.336401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:45:29.336401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:45:29.336401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:45:29.336401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:45:29.336401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:45:29.336401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:29.339255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 515/1037] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jul 11 20:45:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:29.344064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b540ba-18df-4ed5-9745-e537e2dcfd48 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:29.412135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40d18e-7f92-4cbc-8b97-1dfee6e70c4a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:29.450555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71c6c9-9888-4fa2-adf8-4ee871a8440c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:29.451488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71c6c9-9888-4fa2-adf8-4ee871a8440c None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:29.451764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71c6c9-9888-4fa2-adf8-4ee871a8440c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:29.452040 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71c6c9-9888-4fa2-adf8-4ee871a8440c None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:29.452798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71c6c9-9888-4fa2-adf8-4ee871a8440c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:29.454229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71c6c9-9888-4fa2-adf8-4ee871a8440c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:29.474058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b540ba-18df-4ed5-9745-e537e2dcfd48 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:29.476568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 523/1038] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:45:29 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:45:29.528352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04590e-f9e4-4faa-84c2-60b8e34a5603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:29.529060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04590e-f9e4-4faa-84c2-60b8e34a5603 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:29.529342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04590e-f9e4-4faa-84c2-60b8e34a5603 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:29.529617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04590e-f9e4-4faa-84c2-60b8e34a5603 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:29.530498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04590e-f9e4-4faa-84c2-60b8e34a5603 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:29.530809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04590e-f9e4-4faa-84c2-60b8e34a5603 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:29.532349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04590e-f9e4-4faa-84c2-60b8e34a5603 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:29.536522 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:29.559483 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 516/1039] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jul 11 20:45:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:29.561140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71c6c9-9888-4fa2-adf8-4ee871a8440c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:29.577798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:29.680594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71c6c9-9888-4fa2-adf8-4ee871a8440c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:29.718089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84e9d1-b3da-47e2-b529-a5af95a4c938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:29.718757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84e9d1-b3da-47e2-b529-a5af95a4c938 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:29.719039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84e9d1-b3da-47e2-b529-a5af95a4c938 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:29.719314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84e9d1-b3da-47e2-b529-a5af95a4c938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:29.801142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84e9d1-b3da-47e2-b529-a5af95a4c938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:29.802591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84e9d1-b3da-47e2-b529-a5af95a4c938 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:29.814396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:29.910904 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 524/1040] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:45:29.916492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04590e-f9e4-4faa-84c2-60b8e34a5603 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:30.019988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04590e-f9e4-4faa-84c2-60b8e34a5603 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:30.054204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a5855-fbbf-4cf4-9cc7-909860e72be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:30.055010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a5855-fbbf-4cf4-9cc7-909860e72be0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:30.055283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a5855-fbbf-4cf4-9cc7-909860e72be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:30.055550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a5855-fbbf-4cf4-9cc7-909860e72be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:30.065759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a5855-fbbf-4cf4-9cc7-909860e72be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:30.067462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a5855-fbbf-4cf4-9cc7-909860e72be0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-615a5855-fbbf-4cf4-9cc7-909860e72be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:30.085670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:30.086593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:30.087464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:30.087464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:30.087464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:30.087464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:30.091560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 517/1041] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:30.092637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84e9d1-b3da-47e2-b529-a5af95a4c938 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:30.196922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84e9d1-b3da-47e2-b529-a5af95a4c938 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:30.209096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:30.234343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9ddef-e6c7-4823-9e48-ef8c86f836c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:30.235009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9ddef-e6c7-4823-9e48-ef8c86f836c4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:30.235287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9ddef-e6c7-4823-9e48-ef8c86f836c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:30.235557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9ddef-e6c7-4823-9e48-ef8c86f836c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:30.236314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9ddef-e6c7-4823-9e48-ef8c86f836c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:30.236618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9ddef-e6c7-4823-9e48-ef8c86f836c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:30.238040 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9ddef-e6c7-4823-9e48-ef8c86f836c4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:30.249738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 525/1042] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:30.255299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a5855-fbbf-4cf4-9cc7-909860e72be0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:30.302597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:30.356192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a5855-fbbf-4cf4-9cc7-909860e72be0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:30.464861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c03ed-b731-4caa-946f-e4ce99471bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:30.464861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c03ed-b731-4caa-946f-e4ce99471bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:30.464861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c03ed-b731-4caa-946f-e4ce99471bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:30.464861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c03ed-b731-4caa-946f-e4ce99471bd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:30.481216 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c03ed-b731-4caa-946f-e4ce99471bd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:30.482780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c03ed-b731-4caa-946f-e4ce99471bd8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-5c9c03ed-b731-4caa-946f-e4ce99471bd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:30.497747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:30.498694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:30.499580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:30.499580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:30.499580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:30.499580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:30.499580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:30.499580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:30.499580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:30.499580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:30.504530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 518/1043] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:30.505649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9ddef-e6c7-4823-9e48-ef8c86f836c4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:30.612778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9ddef-e6c7-4823-9e48-ef8c86f836c4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:30.646024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a90c5d-3c8d-44cf-a6ec-f5ae2679a373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:30.646690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a90c5d-3c8d-44cf-a6ec-f5ae2679a373 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:30.646968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a90c5d-3c8d-44cf-a6ec-f5ae2679a373 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:30.647239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a90c5d-3c8d-44cf-a6ec-f5ae2679a373 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:30.648025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a90c5d-3c8d-44cf-a6ec-f5ae2679a373 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:30.648359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a90c5d-3c8d-44cf-a6ec-f5ae2679a373 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:30.649789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a90c5d-3c8d-44cf-a6ec-f5ae2679a373 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:30.670160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 526/1044] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:45:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:30.677853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c03ed-b731-4caa-946f-e4ce99471bd8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:30.786748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c03ed-b731-4caa-946f-e4ce99471bd8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:30.793801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:30.830655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b621e0f-b057-452a-b9e0-270492232936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:30.830655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b621e0f-b057-452a-b9e0-270492232936 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:30.830655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b621e0f-b057-452a-b9e0-270492232936 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:30.830655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b621e0f-b057-452a-b9e0-270492232936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:30.831069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b621e0f-b057-452a-b9e0-270492232936 None admin] RBAC: Adding query filter params (role.id=f37ae457d604457bbe6e5dd28ce225e2, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=acb3ad858e4848b9a325c72c7328c863) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:30.831069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b621e0f-b057-452a-b9e0-270492232936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:30.831916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b621e0f-b057-452a-b9e0-270492232936 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:30.844151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 519/1045] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:30 2022] GET /identity/v3/role_assignments?user.id=acb3ad858e4848b9a325c72c7328c863&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=f37ae457d604457bbe6e5dd28ce225e2 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:31.197738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb1943-bd23-4c23-82b8-397aa11ed7cc None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:31.198485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb1943-bd23-4c23-82b8-397aa11ed7cc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:31.198485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb1943-bd23-4c23-82b8-397aa11ed7cc None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:31.201223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 527/1046] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:31.207237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bc476-2549-4aee-ab19-26e4c999dece None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:31.207922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bc476-2549-4aee-ab19-26e4c999dece None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:31.208050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bc476-2549-4aee-ab19-26e4c999dece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:31.227470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-454bc476-2549-4aee-ab19-26e4c999dece None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:31.228707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-454bc476-2549-4aee-ab19-26e4c999dece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:31.306215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 520/1047] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:31.330646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb1943-bd23-4c23-82b8-397aa11ed7cc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:31.364101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:31.448608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb1943-bd23-4c23-82b8-397aa11ed7cc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:31.494745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa41f98-766d-476b-bf35-8e532f551261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:31.495539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa41f98-766d-476b-bf35-8e532f551261 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:31.495822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa41f98-766d-476b-bf35-8e532f551261 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:31.496099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa41f98-766d-476b-bf35-8e532f551261 None admin] PATH_INFO: `/v3/services/image` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:31.496789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa41f98-766d-476b-bf35-8e532f551261 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:31.498183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa41f98-766d-476b-bf35-8e532f551261 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-afa41f98-766d-476b-bf35-8e532f551261 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:31.510972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:31.511998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 20:45:31.513995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:31.516242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 528/1048] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Jul 11 20:45:31 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:31.519586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bc476-2549-4aee-ab19-26e4c999dece None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:31.632728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:31.636393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bc476-2549-4aee-ab19-26e4c999dece None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:31.667457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abe4ff-2d41-4a05-a1b8-ff81320ca49e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:31.667739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abe4ff-2d41-4a05-a1b8-ff81320ca49e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:31.668016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abe4ff-2d41-4a05-a1b8-ff81320ca49e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:31.669449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 529/1049] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:31.674975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ceb2c3-eead-40c2-88a1-efab13193cbc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:31.675273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ceb2c3-eead-40c2-88a1-efab13193cbc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:31.675692 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ceb2c3-eead-40c2-88a1-efab13193cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:31.677108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814ae15-0dd2-4c3c-b774-d2ed03a914a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:31.677886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ae15-0dd2-4c3c-b774-d2ed03a914a2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:31.678170 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ae15-0dd2-4c3c-b774-d2ed03a914a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:31.678445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ae15-0dd2-4c3c-b774-d2ed03a914a2 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:31.679174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ae15-0dd2-4c3c-b774-d2ed03a914a2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:31.679485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ae15-0dd2-4c3c-b774-d2ed03a914a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:31.680936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ae15-0dd2-4c3c-b774-d2ed03a914a2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:31.687678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 521/1050] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:31.693850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814ae15-0dd2-4c3c-b774-d2ed03a914a2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:31.696714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-10ceb2c3-eead-40c2-88a1-efab13193cbc None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:31.698254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-10ceb2c3-eead-40c2-88a1-efab13193cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:31.780217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 530/1051] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:31.796402 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ceb2c3-eead-40c2-88a1-efab13193cbc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:31.806741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814ae15-0dd2-4c3c-b774-d2ed03a914a2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:31.832346 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:31.857261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19623beb-5e50-4b96-9841-90491e4a2aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:31.858499 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19623beb-5e50-4b96-9841-90491e4a2aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:31.859051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19623beb-5e50-4b96-9841-90491e4a2aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:31.859448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19623beb-5e50-4b96-9841-90491e4a2aa6 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:31.860851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19623beb-5e50-4b96-9841-90491e4a2aa6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:31.861293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19623beb-5e50-4b96-9841-90491e4a2aa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:31.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19623beb-5e50-4b96-9841-90491e4a2aa6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:31.874056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 522/1052] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:31.880417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19623beb-5e50-4b96-9841-90491e4a2aa6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:31.913167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:31.914966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ceb2c3-eead-40c2-88a1-efab13193cbc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:31.952630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4c9f5-9882-4433-9e3c-6f05bfffacb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:31.953795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4c9f5-9882-4433-9e3c-6f05bfffacb1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:31.954238 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4c9f5-9882-4433-9e3c-6f05bfffacb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:31.954516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4c9f5-9882-4433-9e3c-6f05bfffacb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:31.967939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4c9f5-9882-4433-9e3c-6f05bfffacb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-cff4c9f5-9882-4433-9e3c-6f05bfffacb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:31.975757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:31.976723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:31.978265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 531/1053] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:45:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:31.982906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4c9f5-9882-4433-9e3c-6f05bfffacb1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:31.999679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19623beb-5e50-4b96-9841-90491e4a2aa6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:32.036968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fee46c-85f5-4b77-9646-34de4bd7f257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:32.037551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fee46c-85f5-4b77-9646-34de4bd7f257 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:32.037956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fee46c-85f5-4b77-9646-34de4bd7f257 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:32.038248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fee46c-85f5-4b77-9646-34de4bd7f257 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:32.039020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fee46c-85f5-4b77-9646-34de4bd7f257 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:32.040697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fee46c-85f5-4b77-9646-34de4bd7f257 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:32.072448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 523/1054] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jul 11 20:45:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:32.085104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:32.104873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4c9f5-9882-4433-9e3c-6f05bfffacb1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:32.151039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1cd3b-40f1-41e1-94d1-77f5fde70601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:32.151752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1cd3b-40f1-41e1-94d1-77f5fde70601 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:32.152005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1cd3b-40f1-41e1-94d1-77f5fde70601 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:32.152302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1cd3b-40f1-41e1-94d1-77f5fde70601 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:32.153092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1cd3b-40f1-41e1-94d1-77f5fde70601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:32.153394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1cd3b-40f1-41e1-94d1-77f5fde70601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:32.154795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1cd3b-40f1-41e1-94d1-77f5fde70601 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:32.165188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 532/1055] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:45:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:32.171134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fee46c-85f5-4b77-9646-34de4bd7f257 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:32.263662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:32.282230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fee46c-85f5-4b77-9646-34de4bd7f257 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:32.317405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732034a-a649-4ec7-a6d8-c21ce82884a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:32.318226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732034a-a649-4ec7-a6d8-c21ce82884a1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:32.318507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732034a-a649-4ec7-a6d8-c21ce82884a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:32.318781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732034a-a649-4ec7-a6d8-c21ce82884a1 None admin] PATH_INFO: `/v3/users/58b7b5795d674bd9be9751983e6171f4` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:32.333206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732034a-a649-4ec7-a6d8-c21ce82884a1 None admin] RBAC: Authorizing `identity:get_user(user_id=58b7b5795d674bd9be9751983e6171f4)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:32.334667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732034a-a649-4ec7-a6d8-c21ce82884a1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:32.336621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 524/1056] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:45:32 2022] GET /identity/v3/users/58b7b5795d674bd9be9751983e6171f4 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:32.342246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1cd3b-40f1-41e1-94d1-77f5fde70601 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:32.448624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1cd3b-40f1-41e1-94d1-77f5fde70601 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:32.485785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb222d-5004-4e89-bf72-4714f0846562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:32.486694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb222d-5004-4e89-bf72-4714f0846562 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:32.487145 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb222d-5004-4e89-bf72-4714f0846562 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:32.487566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb222d-5004-4e89-bf72-4714f0846562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:32.501236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:32.574967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb222d-5004-4e89-bf72-4714f0846562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:32.576361 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb222d-5004-4e89-bf72-4714f0846562 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:32.676326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 533/1057] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:45:32.682140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732034a-a649-4ec7-a6d8-c21ce82884a1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:32.795001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732034a-a649-4ec7-a6d8-c21ce82884a1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:32.839889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924b66e-f3ad-4ed9-a85a-d6ea1d03793b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:32.840791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924b66e-f3ad-4ed9-a85a-d6ea1d03793b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:32.841068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924b66e-f3ad-4ed9-a85a-d6ea1d03793b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:32.841342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924b66e-f3ad-4ed9-a85a-d6ea1d03793b None admin] PATH_INFO: `/v3/projects/a8458669d0a74badb671347fca61e535` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:32.842572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924b66e-f3ad-4ed9-a85a-d6ea1d03793b None admin] RBAC: Authorizing `identity:get_project(project_id=a8458669d0a74badb671347fca61e535)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:32.844013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924b66e-f3ad-4ed9-a85a-d6ea1d03793b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:32.845934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 525/1058] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:45:32 2022] GET /identity/v3/projects/a8458669d0a74badb671347fca61e535 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:32.852091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb222d-5004-4e89-bf72-4714f0846562 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:32.871987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bb101-9881-4009-97c0-d0ae1fbc9056 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:32.872327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bb101-9881-4009-97c0-d0ae1fbc9056 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:32.872563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bb101-9881-4009-97c0-d0ae1fbc9056 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:32.874073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 526/1059] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:45:32.878921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a7b0c-99c5-4c4c-8350-a30eae7f1e8b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:32.879207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a7b0c-99c5-4c4c-8350-a30eae7f1e8b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:32.879478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a7b0c-99c5-4c4c-8350-a30eae7f1e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:32.897794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-712a7b0c-99c5-4c4c-8350-a30eae7f1e8b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:32.898327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:32.899377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-712a7b0c-99c5-4c4c-8350-a30eae7f1e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:32.903054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:32.961179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb222d-5004-4e89-bf72-4714f0846562 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:32.973339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 527/1060] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:32.995889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a7b0c-99c5-4c4c-8350-a30eae7f1e8b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:33.007376 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c38b6-7b84-4317-be67-3fd5b9157a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:33.008824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c38b6-7b84-4317-be67-3fd5b9157a1c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:33.009430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c38b6-7b84-4317-be67-3fd5b9157a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:33.010042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c38b6-7b84-4317-be67-3fd5b9157a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:33.011813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c38b6-7b84-4317-be67-3fd5b9157a1c None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=a8458669d0a74badb671347fca61e535, user.id=58b7b5795d674bd9be9751983e6171f4) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:33.012509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c38b6-7b84-4317-be67-3fd5b9157a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:33.015239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c38b6-7b84-4317-be67-3fd5b9157a1c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:33.024832 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 534/1061] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:32 2022] GET /identity/v3/role_assignments?user.id=58b7b5795d674bd9be9751983e6171f4&scope.project.id=a8458669d0a74badb671347fca61e535&role.id=60bd1c44ce154338a7face8949d07281 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:33.113849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a7b0c-99c5-4c4c-8350-a30eae7f1e8b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:33.152735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8477d6a-1ff3-41b2-93c9-7ce153b06ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:33.153580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8477d6a-1ff3-41b2-93c9-7ce153b06ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:33.153860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8477d6a-1ff3-41b2-93c9-7ce153b06ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:33.154132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8477d6a-1ff3-41b2-93c9-7ce153b06ce8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:33.163418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:33.167616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8477d6a-1ff3-41b2-93c9-7ce153b06ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d8477d6a-1ff3-41b2-93c9-7ce153b06ce8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:33.177255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:33.178255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:33.179086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:33.184058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 528/1062] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jul 11 20:45:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:33.185165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c38b6-7b84-4317-be67-3fd5b9157a1c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:33.292703 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c38b6-7b84-4317-be67-3fd5b9157a1c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:33.300202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:33.329274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d95ad3-65ff-4bb2-b46b-093c33550848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:33.329954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d95ad3-65ff-4bb2-b46b-093c33550848 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:33.330235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d95ad3-65ff-4bb2-b46b-093c33550848 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:33.330508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d95ad3-65ff-4bb2-b46b-093c33550848 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:33.331279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d95ad3-65ff-4bb2-b46b-093c33550848 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:33.331577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d95ad3-65ff-4bb2-b46b-093c33550848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:33.333197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d95ad3-65ff-4bb2-b46b-093c33550848 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:33.341913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 535/1063] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jul 11 20:45:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:33.347857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8477d6a-1ff3-41b2-93c9-7ce153b06ce8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:33.452786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8477d6a-1ff3-41b2-93c9-7ce153b06ce8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:33.486460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8a613-81a0-43a8-9291-7cc4e30359a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:33.487258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8a613-81a0-43a8-9291-7cc4e30359a5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:33.487547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8a613-81a0-43a8-9291-7cc4e30359a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:33.487851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8a613-81a0-43a8-9291-7cc4e30359a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:33.500698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8a613-81a0-43a8-9291-7cc4e30359a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:33.502219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8a613-81a0-43a8-9291-7cc4e30359a5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b9f8a613-81a0-43a8-9291-7cc4e30359a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:33.520809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:33.521790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:33.522677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:33.522677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:33.522677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:33.522677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:33.524443 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 529/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:33.528049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d95ad3-65ff-4bb2-b46b-093c33550848 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:33.631898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d95ad3-65ff-4bb2-b46b-093c33550848 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:33.665748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8601d65-b37e-475f-a3ba-89e896560cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:33.666417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8601d65-b37e-475f-a3ba-89e896560cac None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:33.666700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8601d65-b37e-475f-a3ba-89e896560cac None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:33.666975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8601d65-b37e-475f-a3ba-89e896560cac None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:33.667700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8601d65-b37e-475f-a3ba-89e896560cac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:33.668005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8601d65-b37e-475f-a3ba-89e896560cac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:33.669422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8601d65-b37e-475f-a3ba-89e896560cac None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:33.686270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 536/1065] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:33.687527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8a613-81a0-43a8-9291-7cc4e30359a5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:33.744097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:33.793517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8a613-81a0-43a8-9291-7cc4e30359a5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:33.837448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6808a1-d0f3-4841-8852-a6d89707cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:33.838294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6808a1-d0f3-4841-8852-a6d89707cb09 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:33.838572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6808a1-d0f3-4841-8852-a6d89707cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:33.838843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6808a1-d0f3-4841-8852-a6d89707cb09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:33.846813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6808a1-d0f3-4841-8852-a6d89707cb09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:33.848416 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6808a1-d0f3-4841-8852-a6d89707cb09 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-1c6808a1-d0f3-4841-8852-a6d89707cb09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:45:33.862930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:33.863958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:33.865357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:33.865357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:33.865357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:45:33.865357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:45:33.865357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:45:33.865357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:45:33.865357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:45:33.865357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:45:33.865357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:45:33.865357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:33.868448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 530/1066] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jul 11 20:45:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:33.871027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8601d65-b37e-475f-a3ba-89e896560cac None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:33.900761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:33.974637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8601d65-b37e-475f-a3ba-89e896560cac None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:34.008970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9ce84-f8ef-45ab-aaab-b503ce6cb20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:34.009573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9ce84-f8ef-45ab-aaab-b503ce6cb20d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:34.009860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9ce84-f8ef-45ab-aaab-b503ce6cb20d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:34.010133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9ce84-f8ef-45ab-aaab-b503ce6cb20d None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:34.010968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9ce84-f8ef-45ab-aaab-b503ce6cb20d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:34.011278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9ce84-f8ef-45ab-aaab-b503ce6cb20d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:34.012758 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9ce84-f8ef-45ab-aaab-b503ce6cb20d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:34.035066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 537/1067] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jul 11 20:45:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:34.036596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6808a1-d0f3-4841-8852-a6d89707cb09 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:34.143880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6808a1-d0f3-4841-8852-a6d89707cb09 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:34.157880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:34.180565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a9ca9-56fc-4fc6-8850-d47df0afb713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:34.181369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a9ca9-56fc-4fc6-8850-d47df0afb713 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:34.181644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a9ca9-56fc-4fc6-8850-d47df0afb713 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:34.181908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a9ca9-56fc-4fc6-8850-d47df0afb713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:34.189720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a76a-a008-4ee4-9aff-fd292172722f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:34.190013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a76a-a008-4ee4-9aff-fd292172722f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:34.190290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a76a-a008-4ee4-9aff-fd292172722f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:34.192585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 538/1068] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:34.192812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a9ca9-56fc-4fc6-8850-d47df0afb713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:34.194266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a9ca9-56fc-4fc6-8850-d47df0afb713 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:34.198107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4aa766-ef0c-4ac7-84d7-4a7b9acf7e02 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:34.198107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4aa766-ef0c-4ac7-84d7-4a7b9acf7e02 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:34.199409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4aa766-ef0c-4ac7-84d7-4a7b9acf7e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-8d6a9ca9-56fc-4fc6-8850-d47df0afb713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:34.215019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:34.215963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:34.221734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:34.221734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:34.221734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:34.221734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:34.221734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 531/1069] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:34.222085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5f4aa766-ef0c-4ac7-84d7-4a7b9acf7e02 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:34.222085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5f4aa766-ef0c-4ac7-84d7-4a7b9acf7e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:34.223635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a9ca9-56fc-4fc6-8850-d47df0afb713 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:34.290737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 539/1070] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:34.312667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4aa766-ef0c-4ac7-84d7-4a7b9acf7e02 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:34.327945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a9ca9-56fc-4fc6-8850-d47df0afb713 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:34.331734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:34.366715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe600b7-00cd-494c-8e0a-01ec039d941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:34.367383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe600b7-00cd-494c-8e0a-01ec039d941a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:34.367669 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe600b7-00cd-494c-8e0a-01ec039d941a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:34.367944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe600b7-00cd-494c-8e0a-01ec039d941a None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:34.368706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe600b7-00cd-494c-8e0a-01ec039d941a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:34.369011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe600b7-00cd-494c-8e0a-01ec039d941a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:34.370384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe600b7-00cd-494c-8e0a-01ec039d941a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:34.382078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 532/1071] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:34.387862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe600b7-00cd-494c-8e0a-01ec039d941a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:34.414586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:34.431390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4aa766-ef0c-4ac7-84d7-4a7b9acf7e02 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:34.468704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d898de-7ac5-4d01-b1cd-47e90754377d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:34.469362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d898de-7ac5-4d01-b1cd-47e90754377d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:34.469644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d898de-7ac5-4d01-b1cd-47e90754377d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:34.469917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d898de-7ac5-4d01-b1cd-47e90754377d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:34.501431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe600b7-00cd-494c-8e0a-01ec039d941a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:34.539846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999efe94-daa4-49fb-8f9e-268527d24b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:34.540096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d898de-7ac5-4d01-b1cd-47e90754377d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:34.540689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999efe94-daa4-49fb-8f9e-268527d24b25 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:34.541004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999efe94-daa4-49fb-8f9e-268527d24b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:34.541227 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999efe94-daa4-49fb-8f9e-268527d24b25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:34.541401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d898de-7ac5-4d01-b1cd-47e90754377d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:34.548321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999efe94-daa4-49fb-8f9e-268527d24b25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:34.549757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999efe94-daa4-49fb-8f9e-268527d24b25 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:34.559683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-999efe94-daa4-49fb-8f9e-268527d24b25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:34.563707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:34.565130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:34.566295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:34.566295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:34.566295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:34.566295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:34.566295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:34.566295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:34.566295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:34.566295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:34.566295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 533/1072] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:34.571219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999efe94-daa4-49fb-8f9e-268527d24b25 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:34.650904 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 540/1073] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 20:45:34.656661 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d898de-7ac5-4d01-b1cd-47e90754377d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:34.664525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:34.682369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999efe94-daa4-49fb-8f9e-268527d24b25 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:34.723555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6f870-f236-483f-ad8d-80018dbe90fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:34.724378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6f870-f236-483f-ad8d-80018dbe90fe None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:34.724733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6f870-f236-483f-ad8d-80018dbe90fe None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:34.725084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6f870-f236-483f-ad8d-80018dbe90fe None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:34.725963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6f870-f236-483f-ad8d-80018dbe90fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:34.726348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6f870-f236-483f-ad8d-80018dbe90fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:34.727873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6f870-f236-483f-ad8d-80018dbe90fe None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:34.746092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 534/1074] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:45:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:34.747641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6f870-f236-483f-ad8d-80018dbe90fe None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:34.768071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d898de-7ac5-4d01-b1cd-47e90754377d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:34.785728 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:34.806036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae358-2101-48c2-820c-0428b8f48a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:34.806774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae358-2101-48c2-820c-0428b8f48a36 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:34.807054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae358-2101-48c2-820c-0428b8f48a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:34.807326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae358-2101-48c2-820c-0428b8f48a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:34.818506 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae358-2101-48c2-820c-0428b8f48a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:34.820053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae358-2101-48c2-820c-0428b8f48a36 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-15fae358-2101-48c2-820c-0428b8f48a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:34.842921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:34.844020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:34.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:34.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:34.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:34.845250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:34.848377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 541/1075] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:34.849966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae358-2101-48c2-820c-0428b8f48a36 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:34.868100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6f870-f236-483f-ad8d-80018dbe90fe None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:34.904354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19761078-c18e-4a04-97b7-fcf0cb30f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:34.905678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761078-c18e-4a04-97b7-fcf0cb30f1d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:34.905963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761078-c18e-4a04-97b7-fcf0cb30f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:34.906239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761078-c18e-4a04-97b7-fcf0cb30f1d4 None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/acb3ad858e4848b9a325c72c7328c863/roles/f37ae457d604457bbe6e5dd28ce225e2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:34.908540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19761078-c18e-4a04-97b7-fcf0cb30f1d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=acb3ad858e4848b9a325c72c7328c863, role_id=f37ae457d604457bbe6e5dd28ce225e2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:34.910042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19761078-c18e-4a04-97b7-fcf0cb30f1d4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:34.934858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 535/1076] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:45:34 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/acb3ad858e4848b9a325c72c7328c863/roles/f37ae457d604457bbe6e5dd28ce225e2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:45:34.965121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae358-2101-48c2-820c-0428b8f48a36 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:35.002948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a26f1-23c6-4a23-ad7d-672b397f2ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:35.003795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a26f1-23c6-4a23-ad7d-672b397f2ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:35.004158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a26f1-23c6-4a23-ad7d-672b397f2ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:35.004436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a26f1-23c6-4a23-ad7d-672b397f2ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:35.005279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a26f1-23c6-4a23-ad7d-672b397f2ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:35.005731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a26f1-23c6-4a23-ad7d-672b397f2ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:35.007413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a26f1-23c6-4a23-ad7d-672b397f2ce5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:35.022795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 542/1077] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:35.082397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69435-9afb-4c58-9bfc-3e5a86c8fe26 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:35.082767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69435-9afb-4c58-9bfc-3e5a86c8fe26 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:35.083142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69435-9afb-4c58-9bfc-3e5a86c8fe26 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:35.084804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 536/1078] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:35.091515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06f57f-6b52-45b3-ad44-f0ad8c2e662a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:35.092087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06f57f-6b52-45b3-ad44-f0ad8c2e662a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:35.092407 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06f57f-6b52-45b3-ad44-f0ad8c2e662a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:35.111151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-df06f57f-6b52-45b3-ad44-f0ad8c2e662a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:35.112461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-df06f57f-6b52-45b3-ad44-f0ad8c2e662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:35.145563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:35.200132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 543/1079] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:35.219197 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c69435-9afb-4c58-9bfc-3e5a86c8fe26 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:35.223391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:35.329881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c69435-9afb-4c58-9bfc-3e5a86c8fe26 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:35.364443 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01bf74-890b-4a6e-af05-09683f4cc4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:35.365251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01bf74-890b-4a6e-af05-09683f4cc4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:35.365530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01bf74-890b-4a6e-af05-09683f4cc4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:35.365802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01bf74-890b-4a6e-af05-09683f4cc4b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:35.375779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01bf74-890b-4a6e-af05-09683f4cc4b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-7e01bf74-890b-4a6e-af05-09683f4cc4b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:35.383853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:35.384918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:35.385925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:35.388258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 537/1080] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:45:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:35.391413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06f57f-6b52-45b3-ad44-f0ad8c2e662a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:35.511260 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06f57f-6b52-45b3-ad44-f0ad8c2e662a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:35.548348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cd806-14bd-4cf8-9041-840e70a823fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:35.548978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd806-14bd-4cf8-9041-840e70a823fd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:35.549269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd806-14bd-4cf8-9041-840e70a823fd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:35.549535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd806-14bd-4cf8-9041-840e70a823fd None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:35.550368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cd806-14bd-4cf8-9041-840e70a823fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:35.550657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cd806-14bd-4cf8-9041-840e70a823fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:35.552512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cd806-14bd-4cf8-9041-840e70a823fd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:35.560884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 544/1081] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:45:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:35.566935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01bf74-890b-4a6e-af05-09683f4cc4b3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:35.632992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:35.671578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01bf74-890b-4a6e-af05-09683f4cc4b3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:35.706535 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcf7d9-3e40-4946-9447-9e5a045bcbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:35.707628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcf7d9-3e40-4946-9447-9e5a045bcbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:35.707904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcf7d9-3e40-4946-9447-9e5a045bcbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:35.708203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcf7d9-3e40-4946-9447-9e5a045bcbb4 None admin] PATH_INFO: `/v3/users/58b7b5795d674bd9be9751983e6171f4` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:35.709387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcf7d9-3e40-4946-9447-9e5a045bcbb4 None admin] RBAC: Authorizing `identity:get_user(user_id=58b7b5795d674bd9be9751983e6171f4)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:35.710823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcf7d9-3e40-4946-9447-9e5a045bcbb4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:35.712504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 538/1082] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:45:35 2022] GET /identity/v3/users/58b7b5795d674bd9be9751983e6171f4 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:35.718363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cd806-14bd-4cf8-9041-840e70a823fd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:35.741018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:35.826636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cd806-14bd-4cf8-9041-840e70a823fd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:35.864293 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2cf4f-930f-4548-a1f1-303f9d6bb672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:35.865184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2cf4f-930f-4548-a1f1-303f9d6bb672 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:35.865465 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2cf4f-930f-4548-a1f1-303f9d6bb672 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:35.865743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2cf4f-930f-4548-a1f1-303f9d6bb672 None admin] PATH_INFO: `/v3/projects/a8458669d0a74badb671347fca61e535` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:35.866954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2cf4f-930f-4548-a1f1-303f9d6bb672 None admin] RBAC: Authorizing `identity:get_project(project_id=a8458669d0a74badb671347fca61e535)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:35.868456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2cf4f-930f-4548-a1f1-303f9d6bb672 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:35.870324 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 545/1083] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:45:35 2022] GET /identity/v3/projects/a8458669d0a74badb671347fca61e535 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:35.876234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcf7d9-3e40-4946-9447-9e5a045bcbb4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:35.981719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcf7d9-3e40-4946-9447-9e5a045bcbb4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:36.012686 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:36.018663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af3f06-dbd9-461f-b289-cc54b7341bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:36.020126 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af3f06-dbd9-461f-b289-cc54b7341bfd None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:36.020425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af3f06-dbd9-461f-b289-cc54b7341bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:36.020706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af3f06-dbd9-461f-b289-cc54b7341bfd None admin] PATH_INFO: `/v3/projects/a8458669d0a74badb671347fca61e535/users/58b7b5795d674bd9be9751983e6171f4/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:36.022716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af3f06-dbd9-461f-b289-cc54b7341bfd None admin] RBAC: Authorizing `identity:create_grant(project_id=a8458669d0a74badb671347fca61e535, user_id=58b7b5795d674bd9be9751983e6171f4, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:36.024205 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af3f06-dbd9-461f-b289-cc54b7341bfd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:36.040029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 539/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:45:35 2022] PUT /identity/v3/projects/a8458669d0a74badb671347fca61e535/users/58b7b5795d674bd9be9751983e6171f4/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:45:36.944007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c5a4b-0cae-4f3f-aae9-372c345ac545 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:36.944606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c5a4b-0cae-4f3f-aae9-372c345ac545 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:36.944606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c5a4b-0cae-4f3f-aae9-372c345ac545 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:36.946122 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 546/1085] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:36.951723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ac020-1c18-4130-8ba2-e9d633567b19 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:36.952156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ac020-1c18-4130-8ba2-e9d633567b19 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:36.952379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ac020-1c18-4130-8ba2-e9d633567b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:36.970176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-005ac020-1c18-4130-8ba2-e9d633567b19 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:36.971366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-005ac020-1c18-4130-8ba2-e9d633567b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:37.041474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 540/1086] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:37.065005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c5a4b-0cae-4f3f-aae9-372c345ac545 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:37.162593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:37.177085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c5a4b-0cae-4f3f-aae9-372c345ac545 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:37.213281 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5de5b-8600-4e54-b3ce-56ba2807e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:37.214098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5de5b-8600-4e54-b3ce-56ba2807e0dc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:37.214380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5de5b-8600-4e54-b3ce-56ba2807e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:37.214653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5de5b-8600-4e54-b3ce-56ba2807e0dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:37.226323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5de5b-8600-4e54-b3ce-56ba2807e0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b5c5de5b-8600-4e54-b3ce-56ba2807e0dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:37.234542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:37.235559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:37.236454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 547/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jul 11 20:45:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:37.242481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ac020-1c18-4130-8ba2-e9d633567b19 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:37.343171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ac020-1c18-4130-8ba2-e9d633567b19 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:37.365680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:37.378166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18be04e-1a95-41ea-85a9-7bec1a47d1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:37.378883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18be04e-1a95-41ea-85a9-7bec1a47d1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:37.379159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18be04e-1a95-41ea-85a9-7bec1a47d1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:37.379429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18be04e-1a95-41ea-85a9-7bec1a47d1c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:37.380205 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18be04e-1a95-41ea-85a9-7bec1a47d1c0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:37.380510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18be04e-1a95-41ea-85a9-7bec1a47d1c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:37.381895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18be04e-1a95-41ea-85a9-7bec1a47d1c0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:37.389751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 541/1088] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jul 11 20:45:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:37.395478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5de5b-8600-4e54-b3ce-56ba2807e0dc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:37.498552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5de5b-8600-4e54-b3ce-56ba2807e0dc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:37.533716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576e2d1-4e5c-4242-b560-0327b459caaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:37.534720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576e2d1-4e5c-4242-b560-0327b459caaf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:37.534880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576e2d1-4e5c-4242-b560-0327b459caaf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:37.535122 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576e2d1-4e5c-4242-b560-0327b459caaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:37.545081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576e2d1-4e5c-4242-b560-0327b459caaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:37.546544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576e2d1-4e5c-4242-b560-0327b459caaf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:37.555169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b576e2d1-4e5c-4242-b560-0327b459caaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:37.565416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:37.566470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:37.567349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:37.567349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:37.567349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:37.567349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:37.567349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 548/1089] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:37.573591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18be04e-1a95-41ea-85a9-7bec1a47d1c0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:37.677792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18be04e-1a95-41ea-85a9-7bec1a47d1c0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:37.714250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32756d-2422-4a05-8957-12398ca937fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:37.714917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32756d-2422-4a05-8957-12398ca937fe None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:37.715198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32756d-2422-4a05-8957-12398ca937fe None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:37.715462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32756d-2422-4a05-8957-12398ca937fe None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:37.716305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32756d-2422-4a05-8957-12398ca937fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:37.716617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32756d-2422-4a05-8957-12398ca937fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:37.717997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32756d-2422-4a05-8957-12398ca937fe None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:37.731232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 542/1090] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:37.737008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576e2d1-4e5c-4242-b560-0327b459caaf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:37.844804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576e2d1-4e5c-4242-b560-0327b459caaf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:37.881834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e30e1-0672-49e6-94a7-081b11f85741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:37.882654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e30e1-0672-49e6-94a7-081b11f85741 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:37.882953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e30e1-0672-49e6-94a7-081b11f85741 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:37.883232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e30e1-0672-49e6-94a7-081b11f85741 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:37.890589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e30e1-0672-49e6-94a7-081b11f85741 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:37.892041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e30e1-0672-49e6-94a7-081b11f85741 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-343e30e1-0672-49e6-94a7-081b11f85741 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:45:37.908351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:37.909339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:37.910288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 549/1091] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jul 11 20:45:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:37.916018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32756d-2422-4a05-8957-12398ca937fe None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:37.964978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:38.019641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32756d-2422-4a05-8957-12398ca937fe None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:38.049181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ee91-a454-447f-b757-1a2a202bdeb7 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.049677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ee91-a454-447f-b757-1a2a202bdeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.049919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ee91-a454-447f-b757-1a2a202bdeb7 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.051359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 550/1092] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:38.053519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7102d-5132-431e-b9d8-15b8d9b8123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:38.054195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7102d-5132-431e-b9d8-15b8d9b8123e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.054473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7102d-5132-431e-b9d8-15b8d9b8123e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.054743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7102d-5132-431e-b9d8-15b8d9b8123e None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.055579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7102d-5132-431e-b9d8-15b8d9b8123e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:38.055884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7102d-5132-431e-b9d8-15b8d9b8123e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:38.056553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d05b1-f1b3-4615-8267-af283390fa27 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.056838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d05b1-f1b3-4615-8267-af283390fa27 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.057110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d05b1-f1b3-4615-8267-af283390fa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.057343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7102d-5132-431e-b9d8-15b8d9b8123e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:38.074509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b43d05b1-f1b3-4615-8267-af283390fa27 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:38.075214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 543/1093] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jul 11 20:45:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:38.076817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b43d05b1-f1b3-4615-8267-af283390fa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:38.080827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7102d-5132-431e-b9d8-15b8d9b8123e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:38.131840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:38.143740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 551/1094] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:38.167462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d05b1-f1b3-4615-8267-af283390fa27 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:38.179345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7102d-5132-431e-b9d8-15b8d9b8123e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:38.180908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:38.216685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba33193-067a-4112-a218-8536d224c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:38.217328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba33193-067a-4112-a218-8536d224c34c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.217603 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba33193-067a-4112-a218-8536d224c34c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.217888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba33193-067a-4112-a218-8536d224c34c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.272091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d05b1-f1b3-4615-8267-af283390fa27 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:38.289407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba33193-067a-4112-a218-8536d224c34c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:38.290882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba33193-067a-4112-a218-8536d224c34c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:38.306150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c32a90-9e38-4b70-9f48-9460e7478749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:38.306954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c32a90-9e38-4b70-9f48-9460e7478749 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.307237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c32a90-9e38-4b70-9f48-9460e7478749 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.307508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c32a90-9e38-4b70-9f48-9460e7478749 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.318708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c32a90-9e38-4b70-9f48-9460e7478749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d5c32a90-9e38-4b70-9f48-9460e7478749 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:38.326691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:38.327622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:45:38.328491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 552/1095] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:45:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:38.333949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c32a90-9e38-4b70-9f48-9460e7478749 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:38.382816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:38.395497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 544/1096] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 20:45:38.401462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba33193-067a-4112-a218-8536d224c34c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:38.403351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:38.433855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c32a90-9e38-4b70-9f48-9460e7478749 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:38.469310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b678820-f8e9-49e9-b072-9bf49050a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:38.469998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b678820-f8e9-49e9-b072-9bf49050a04d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.470283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b678820-f8e9-49e9-b072-9bf49050a04d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.470599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b678820-f8e9-49e9-b072-9bf49050a04d None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.471334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b678820-f8e9-49e9-b072-9bf49050a04d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:38.471640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b678820-f8e9-49e9-b072-9bf49050a04d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:38.473176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b678820-f8e9-49e9-b072-9bf49050a04d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:38.481535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 553/1097] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:45:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:38.487355 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b678820-f8e9-49e9-b072-9bf49050a04d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:38.503706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba33193-067a-4112-a218-8536d224c34c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:38.543306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612f727-cc83-43c3-804d-ce846ab7cc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:38.544131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612f727-cc83-43c3-804d-ce846ab7cc65 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.544417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612f727-cc83-43c3-804d-ce846ab7cc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.544682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612f727-cc83-43c3-804d-ce846ab7cc65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.554959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612f727-cc83-43c3-804d-ce846ab7cc65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:38.556462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612f727-cc83-43c3-804d-ce846ab7cc65 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-0612f727-cc83-43c3-804d-ce846ab7cc65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:45:38.574355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:38.575437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:38.576544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:45:38.576544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:45:38.576544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:45:38.576544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:38.576544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 545/1098] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:38.581168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612f727-cc83-43c3-804d-ce846ab7cc65 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:38.590103 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b678820-f8e9-49e9-b072-9bf49050a04d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:38.603236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:38.624312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386c2e3-74a6-4905-920a-ac155efd7a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:38.625283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386c2e3-74a6-4905-920a-ac155efd7a2d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.625632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386c2e3-74a6-4905-920a-ac155efd7a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.625973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386c2e3-74a6-4905-920a-ac155efd7a2d None admin] PATH_INFO: `/v3/users/58b7b5795d674bd9be9751983e6171f4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.627254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386c2e3-74a6-4905-920a-ac155efd7a2d None admin] RBAC: Authorizing `identity:get_user(user_id=58b7b5795d674bd9be9751983e6171f4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:38.628982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386c2e3-74a6-4905-920a-ac155efd7a2d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:38.630633 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 554/1099] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:45:38 2022] GET /identity/v3/users/58b7b5795d674bd9be9751983e6171f4 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:38.636324 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386c2e3-74a6-4905-920a-ac155efd7a2d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:38.636465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:38.682187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612f727-cc83-43c3-804d-ce846ab7cc65 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:38.717041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2f798-cfcd-4b53-88bf-cdbb6a49953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:38.717708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2f798-cfcd-4b53-88bf-cdbb6a49953f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.717989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2f798-cfcd-4b53-88bf-cdbb6a49953f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.718260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2f798-cfcd-4b53-88bf-cdbb6a49953f None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.718991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2f798-cfcd-4b53-88bf-cdbb6a49953f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:38.719298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2f798-cfcd-4b53-88bf-cdbb6a49953f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:38.720698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2f798-cfcd-4b53-88bf-cdbb6a49953f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:38.732169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 546/1100] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:38.735643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386c2e3-74a6-4905-920a-ac155efd7a2d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:38.737851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2f798-cfcd-4b53-88bf-cdbb6a49953f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:38.769622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed5eb8-9380-40c4-980a-b9d8c2a56941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:38.770223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed5eb8-9380-40c4-980a-b9d8c2a56941 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.770506 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed5eb8-9380-40c4-980a-b9d8c2a56941 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.770784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed5eb8-9380-40c4-980a-b9d8c2a56941 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.820586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:38.843840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed5eb8-9380-40c4-980a-b9d8c2a56941 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:38.845348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed5eb8-9380-40c4-980a-b9d8c2a56941 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:38.847253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2f798-cfcd-4b53-88bf-cdbb6a49953f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:38.863014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:38.890975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab39942-1ac6-4383-9e7d-0e279d609763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:38.891937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab39942-1ac6-4383-9e7d-0e279d609763 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:38.892388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab39942-1ac6-4383-9e7d-0e279d609763 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:38.892801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab39942-1ac6-4383-9e7d-0e279d609763 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:38.902002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab39942-1ac6-4383-9e7d-0e279d609763 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:38.903974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab39942-1ac6-4383-9e7d-0e279d609763 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-aab39942-1ac6-4383-9e7d-0e279d609763 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:45:38.918852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:38.919891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:38.920830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:45:38.920830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:45:38.920830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:45:38.920830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:45:38.920830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:45:38.920830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:45:38.920830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:45:38.920830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:38.920830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 547/1101] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:45:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:45:38.925881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab39942-1ac6-4383-9e7d-0e279d609763 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:38.952388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 555/1102] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 20:45:38.953453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed5eb8-9380-40c4-980a-b9d8c2a56941 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:39.027404 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab39942-1ac6-4383-9e7d-0e279d609763 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:39.032031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:39.051665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed5eb8-9380-40c4-980a-b9d8c2a56941 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:39.060530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c251b-18ae-450e-925b-f86e69ad1bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:39.061193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c251b-18ae-450e-925b-f86e69ad1bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:39.061470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c251b-18ae-450e-925b-f86e69ad1bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:39.061747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c251b-18ae-450e-925b-f86e69ad1bf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:39.062532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c251b-18ae-450e-925b-f86e69ad1bf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:39.062837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c251b-18ae-450e-925b-f86e69ad1bf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:39.064287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c251b-18ae-450e-925b-f86e69ad1bf7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:39.077389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 548/1103] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:45:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:39.083605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c251b-18ae-450e-925b-f86e69ad1bf7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:39.087993 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906b7f9-c380-417e-b3b5-c228c67cf2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:39.088892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906b7f9-c380-417e-b3b5-c228c67cf2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:39.089171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906b7f9-c380-417e-b3b5-c228c67cf2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:39.089442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906b7f9-c380-417e-b3b5-c228c67cf2a8 None admin] PATH_INFO: `/v3/projects/a8458669d0a74badb671347fca61e535` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:39.090640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906b7f9-c380-417e-b3b5-c228c67cf2a8 None admin] RBAC: Authorizing `identity:get_project(project_id=a8458669d0a74badb671347fca61e535)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:39.092084 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906b7f9-c380-417e-b3b5-c228c67cf2a8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:39.093816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 556/1104] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:45:38 2022] GET /identity/v3/projects/a8458669d0a74badb671347fca61e535 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:39.099747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906b7f9-c380-417e-b3b5-c228c67cf2a8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:39.118299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:39.190792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c251b-18ae-450e-925b-f86e69ad1bf7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:39.197833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906b7f9-c380-417e-b3b5-c228c67cf2a8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:39.230345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdccb7-f736-4e2e-8f8a-176b20b8604b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:39.231017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdccb7-f736-4e2e-8f8a-176b20b8604b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:39.231286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdccb7-f736-4e2e-8f8a-176b20b8604b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:39.231577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdccb7-f736-4e2e-8f8a-176b20b8604b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:39.232882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdccb7-f736-4e2e-8f8a-176b20b8604b None admin] RBAC: Adding query filter params (role.id=f37ae457d604457bbe6e5dd28ce225e2, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=acb3ad858e4848b9a325c72c7328c863) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:39.233309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdccb7-f736-4e2e-8f8a-176b20b8604b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:39.234641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760e908-7435-424c-9f5d-e5022de17ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:39.235403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdccb7-f736-4e2e-8f8a-176b20b8604b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:39.235528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760e908-7435-424c-9f5d-e5022de17ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:39.235928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760e908-7435-424c-9f5d-e5022de17ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:39.236331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760e908-7435-424c-9f5d-e5022de17ca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:39.237648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760e908-7435-424c-9f5d-e5022de17ca7 None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=a8458669d0a74badb671347fca61e535, user.id=58b7b5795d674bd9be9751983e6171f4) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:39.238328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760e908-7435-424c-9f5d-e5022de17ca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:39.240646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760e908-7435-424c-9f5d-e5022de17ca7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:39.242690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 549/1105] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:39 2022] GET /identity/v3/role_assignments?user.id=acb3ad858e4848b9a325c72c7328c863&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=f37ae457d604457bbe6e5dd28ce225e2 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:39.249200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 557/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:39 2022] GET /identity/v3/role_assignments?user.id=58b7b5795d674bd9be9751983e6171f4&scope.project.id=a8458669d0a74badb671347fca61e535&role.id=60bd1c44ce154338a7face8949d07281 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:41.327938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daeb25-cb11-43b5-ae75-de1b38960e7e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:41.328955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daeb25-cb11-43b5-ae75-de1b38960e7e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:41.328955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daeb25-cb11-43b5-ae75-de1b38960e7e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:41.331145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 550/1107] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:41.336655 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8bc7e-ad25-4c49-bba7-9f8ddb7b360b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:41.337069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8bc7e-ad25-4c49-bba7-9f8ddb7b360b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:41.337162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8bc7e-ad25-4c49-bba7-9f8ddb7b360b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:41.355770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ecb8bc7e-ad25-4c49-bba7-9f8ddb7b360b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:41.356972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ecb8bc7e-ad25-4c49-bba7-9f8ddb7b360b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:41.364098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:41.426964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 558/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:41.463082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39daeb25-cb11-43b5-ae75-de1b38960e7e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:41.577157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:41.582905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39daeb25-cb11-43b5-ae75-de1b38960e7e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:41.616119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7de7e-5b6b-4950-bc98-01ab83dd72ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:41.616936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7de7e-5b6b-4950-bc98-01ab83dd72ac None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:41.617217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7de7e-5b6b-4950-bc98-01ab83dd72ac None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:41.617498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7de7e-5b6b-4950-bc98-01ab83dd72ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:41.618682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7de7e-5b6b-4950-bc98-01ab83dd72ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:41.620155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7de7e-5b6b-4950-bc98-01ab83dd72ac None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:41.621813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 551/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:41.628603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8bc7e-ad25-4c49-bba7-9f8ddb7b360b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:41.746672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8bc7e-ad25-4c49-bba7-9f8ddb7b360b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:41.782767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d7ed0-f1b8-4123-baeb-217c1d851ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:41.783429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d7ed0-f1b8-4123-baeb-217c1d851ff1 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:41.783656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d7ed0-f1b8-4123-baeb-217c1d851ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:41.783965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d7ed0-f1b8-4123-baeb-217c1d851ff1 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:41.784747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d7ed0-f1b8-4123-baeb-217c1d851ff1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:41.786188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d7ed0-f1b8-4123-baeb-217c1d851ff1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:41.803900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:41.816995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 559/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:45:41 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:43.858898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a3b68-2d34-46e5-a43b-70739d4956ec None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:43.859359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a3b68-2d34-46e5-a43b-70739d4956ec None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:43.859473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a3b68-2d34-46e5-a43b-70739d4956ec None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:43.861517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 552/1111] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:43.867790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a150f2f-130b-4a19-8f99-5196c65207ad None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:43.868253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a150f2f-130b-4a19-8f99-5196c65207ad None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:43.868340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a150f2f-130b-4a19-8f99-5196c65207ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:43.885796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2a150f2f-130b-4a19-8f99-5196c65207ad None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:43.886975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2a150f2f-130b-4a19-8f99-5196c65207ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:43.955602 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 560/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:43.979350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a3b68-2d34-46e5-a43b-70739d4956ec None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:44.099996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a3b68-2d34-46e5-a43b-70739d4956ec None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:44.133293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c36ab-cc74-4bd9-b5d1-a544a0fab35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:44.134143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c36ab-cc74-4bd9-b5d1-a544a0fab35d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:44.134420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c36ab-cc74-4bd9-b5d1-a544a0fab35d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:44.134690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c36ab-cc74-4bd9-b5d1-a544a0fab35d None admin] PATH_INFO: `/v3/roles/e2c9006dc77f473f8e1064f285400a98` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:44.136333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c36ab-cc74-4bd9-b5d1-a544a0fab35d None admin] RBAC: Authorizing `identity:get_role(role_id=e2c9006dc77f473f8e1064f285400a98)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:44.137705 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c36ab-cc74-4bd9-b5d1-a544a0fab35d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:44.139069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 553/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:45:43 2022] GET /identity/v3/roles/e2c9006dc77f473f8e1064f285400a98 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:44.148642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a150f2f-130b-4a19-8f99-5196c65207ad None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:44.264453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a150f2f-130b-4a19-8f99-5196c65207ad None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:44.294948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:44.304567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bc180-ad8b-48ae-85c1-6d60a2893ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:44.305443 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bc180-ad8b-48ae-85c1-6d60a2893ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:44.305726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bc180-ad8b-48ae-85c1-6d60a2893ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:44.305994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bc180-ad8b-48ae-85c1-6d60a2893ab6 None admin] PATH_INFO: `/v3/users/f1b9757e13bb41eeadf9ce49e5e66356` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:44.320536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917bc180-ad8b-48ae-85c1-6d60a2893ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=f1b9757e13bb41eeadf9ce49e5e66356)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:44.322342 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917bc180-ad8b-48ae-85c1-6d60a2893ab6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:44.324232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 561/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:45:44 2022] GET /identity/v3/users/f1b9757e13bb41eeadf9ce49e5e66356 => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:44.330324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c36ab-cc74-4bd9-b5d1-a544a0fab35d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:44.376667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:44.443630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c36ab-cc74-4bd9-b5d1-a544a0fab35d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:44.478829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74acfef-7600-4aed-ba51-8dc079bd4088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:44.479467 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74acfef-7600-4aed-ba51-8dc079bd4088 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:44.479744 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74acfef-7600-4aed-ba51-8dc079bd4088 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:44.480008 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74acfef-7600-4aed-ba51-8dc079bd4088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:44.550325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74acfef-7600-4aed-ba51-8dc079bd4088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:44.551769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74acfef-7600-4aed-ba51-8dc079bd4088 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:44.584994 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:44.656875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 554/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:45:44.663362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bc180-ad8b-48ae-85c1-6d60a2893ab6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:44.777009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bc180-ad8b-48ae-85c1-6d60a2893ab6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:44.812778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55042f4-f7ab-4cef-919c-eafc0ec6622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:44.813633 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55042f4-f7ab-4cef-919c-eafc0ec6622a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:44.813908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55042f4-f7ab-4cef-919c-eafc0ec6622a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:44.814183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55042f4-f7ab-4cef-919c-eafc0ec6622a None admin] PATH_INFO: `/v3/projects/a8458669d0a74badb671347fca61e535` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:44.815412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55042f4-f7ab-4cef-919c-eafc0ec6622a None admin] RBAC: Authorizing `identity:get_project(project_id=a8458669d0a74badb671347fca61e535)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:44.817023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55042f4-f7ab-4cef-919c-eafc0ec6622a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:44.818733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 562/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:45:44 2022] GET /identity/v3/projects/a8458669d0a74badb671347fca61e535 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:44.825395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74acfef-7600-4aed-ba51-8dc079bd4088 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:44.929289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74acfef-7600-4aed-ba51-8dc079bd4088 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:44.968420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8ad65-19d0-4d3e-8ad1-322abf17219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:44.969067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8ad65-19d0-4d3e-8ad1-322abf17219f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:44.969352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8ad65-19d0-4d3e-8ad1-322abf17219f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:44.969624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8ad65-19d0-4d3e-8ad1-322abf17219f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:44.970465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8ad65-19d0-4d3e-8ad1-322abf17219f None admin] RBAC: Adding query filter params (role.id=e2c9006dc77f473f8e1064f285400a98, scope.project.id=a8458669d0a74badb671347fca61e535, user.id=f1b9757e13bb41eeadf9ce49e5e66356) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:44.970776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8ad65-19d0-4d3e-8ad1-322abf17219f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:44.972313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8ad65-19d0-4d3e-8ad1-322abf17219f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:44.974161 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:44.979359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 555/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:44 2022] GET /identity/v3/role_assignments?user.id=f1b9757e13bb41eeadf9ce49e5e66356&scope.project.id=a8458669d0a74badb671347fca61e535&role.id=e2c9006dc77f473f8e1064f285400a98 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:46.973328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7a957-e530-46d6-8e89-c5a00d42f682 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:46.974242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7a957-e530-46d6-8e89-c5a00d42f682 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:46.974242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7a957-e530-46d6-8e89-c5a00d42f682 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:46.976478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 563/1118] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:46.982768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dec5b-46e4-4cc2-a671-2a6a1fe49717 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:46.983540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dec5b-46e4-4cc2-a671-2a6a1fe49717 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:46.983741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dec5b-46e4-4cc2-a671-2a6a1fe49717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:47.003912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5e3dec5b-46e4-4cc2-a671-2a6a1fe49717 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:47.005165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5e3dec5b-46e4-4cc2-a671-2a6a1fe49717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:47.076775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 556/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:47.099723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7a957-e530-46d6-8e89-c5a00d42f682 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:47.163513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:47.208853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7a957-e530-46d6-8e89-c5a00d42f682 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:47.243629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a70c3-dd0b-4426-b7e0-eba80f302b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:47.244510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a70c3-dd0b-4426-b7e0-eba80f302b86 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:47.244789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a70c3-dd0b-4426-b7e0-eba80f302b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:47.245062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a70c3-dd0b-4426-b7e0-eba80f302b86 None admin] PATH_INFO: `/v3/roles/e2c9006dc77f473f8e1064f285400a98` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:47.246645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a70c3-dd0b-4426-b7e0-eba80f302b86 None admin] RBAC: Authorizing `identity:get_role(role_id=e2c9006dc77f473f8e1064f285400a98)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:47.248035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a70c3-dd0b-4426-b7e0-eba80f302b86 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:47.249502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 564/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:45:47 2022] GET /identity/v3/roles/e2c9006dc77f473f8e1064f285400a98 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:47.257051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3dec5b-46e4-4cc2-a671-2a6a1fe49717 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:47.358884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3dec5b-46e4-4cc2-a671-2a6a1fe49717 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:47.385838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:47.395212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68ce1b-80d8-417e-a3af-fa804cc7f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:47.396079 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68ce1b-80d8-417e-a3af-fa804cc7f67e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:47.396400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68ce1b-80d8-417e-a3af-fa804cc7f67e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:47.396667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68ce1b-80d8-417e-a3af-fa804cc7f67e None admin] PATH_INFO: `/v3/users/f1b9757e13bb41eeadf9ce49e5e66356` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:47.397879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68ce1b-80d8-417e-a3af-fa804cc7f67e None admin] RBAC: Authorizing `identity:get_user(user_id=f1b9757e13bb41eeadf9ce49e5e66356)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:47.399315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68ce1b-80d8-417e-a3af-fa804cc7f67e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:47.400990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 557/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:45:47 2022] GET /identity/v3/users/f1b9757e13bb41eeadf9ce49e5e66356 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:47.407138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a70c3-dd0b-4426-b7e0-eba80f302b86 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:47.516241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a70c3-dd0b-4426-b7e0-eba80f302b86 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:47.540164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:47.553280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b83323-0c16-4341-8566-cfb08324b966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:47.554148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b83323-0c16-4341-8566-cfb08324b966 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:47.554692 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b83323-0c16-4341-8566-cfb08324b966 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:47.554974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b83323-0c16-4341-8566-cfb08324b966 None admin] PATH_INFO: `/v3/projects/a8458669d0a74badb671347fca61e535` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:47.556213 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b83323-0c16-4341-8566-cfb08324b966 None admin] RBAC: Authorizing `identity:get_project(project_id=a8458669d0a74badb671347fca61e535)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:47.557650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b83323-0c16-4341-8566-cfb08324b966 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:47.559370 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 565/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:45:47 2022] GET /identity/v3/projects/a8458669d0a74badb671347fca61e535 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:47.565521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68ce1b-80d8-417e-a3af-fa804cc7f67e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:47.669133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68ce1b-80d8-417e-a3af-fa804cc7f67e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:47.704695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb074f-04b9-4cca-b4c8-24ad9639b832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:47.705584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb074f-04b9-4cca-b4c8-24ad9639b832 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:47.705866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb074f-04b9-4cca-b4c8-24ad9639b832 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:47.706172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb074f-04b9-4cca-b4c8-24ad9639b832 None admin] PATH_INFO: `/v3/projects/a8458669d0a74badb671347fca61e535/users/f1b9757e13bb41eeadf9ce49e5e66356/roles/e2c9006dc77f473f8e1064f285400a98` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:47.708856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb074f-04b9-4cca-b4c8-24ad9639b832 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8458669d0a74badb671347fca61e535, user_id=f1b9757e13bb41eeadf9ce49e5e66356, role_id=e2c9006dc77f473f8e1064f285400a98)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:47.710289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb074f-04b9-4cca-b4c8-24ad9639b832 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:47.726293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 558/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:45:47 2022] PUT /identity/v3/projects/a8458669d0a74badb671347fca61e535/users/f1b9757e13bb41eeadf9ce49e5e66356/roles/e2c9006dc77f473f8e1064f285400a98 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:45:49.724246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e9a2b-06e8-4101-ab88-992765c0090e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:49.724846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e9a2b-06e8-4101-ab88-992765c0090e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:49.724846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e9a2b-06e8-4101-ab88-992765c0090e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:49.726169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 566/1124] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:49.732451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae009723-a36b-40ea-b7b0-4e46a7ff5f78 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:49.732900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae009723-a36b-40ea-b7b0-4e46a7ff5f78 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:49.733021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae009723-a36b-40ea-b7b0-4e46a7ff5f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:49.753945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ae009723-a36b-40ea-b7b0-4e46a7ff5f78 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:49.755137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ae009723-a36b-40ea-b7b0-4e46a7ff5f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:49.812607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:49.838669 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 559/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:49.861859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e9a2b-06e8-4101-ab88-992765c0090e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:49.982766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e9a2b-06e8-4101-ab88-992765c0090e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:50.018590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42854a70-97e5-4066-9470-3387a57038c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:50.019511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42854a70-97e5-4066-9470-3387a57038c2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:50.019798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42854a70-97e5-4066-9470-3387a57038c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:50.020071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42854a70-97e5-4066-9470-3387a57038c2 None admin] PATH_INFO: `/v3/roles/e2c9006dc77f473f8e1064f285400a98` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:50.021619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42854a70-97e5-4066-9470-3387a57038c2 None admin] RBAC: Authorizing `identity:get_role(role_id=e2c9006dc77f473f8e1064f285400a98)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:50.023034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42854a70-97e5-4066-9470-3387a57038c2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:50.024431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 567/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:45:49 2022] GET /identity/v3/roles/e2c9006dc77f473f8e1064f285400a98 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:50.033310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae009723-a36b-40ea-b7b0-4e46a7ff5f78 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:50.143532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae009723-a36b-40ea-b7b0-4e46a7ff5f78 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:50.195958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d3a18-4a7a-4404-af39-2a2e733cee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:50.197237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d3a18-4a7a-4404-af39-2a2e733cee32 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:50.197521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d3a18-4a7a-4404-af39-2a2e733cee32 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:50.197796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d3a18-4a7a-4404-af39-2a2e733cee32 None admin] PATH_INFO: `/v3/users/f1b9757e13bb41eeadf9ce49e5e66356` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:50.199125 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d3a18-4a7a-4404-af39-2a2e733cee32 None admin] RBAC: Authorizing `identity:get_user(user_id=f1b9757e13bb41eeadf9ce49e5e66356)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:50.200620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d3a18-4a7a-4404-af39-2a2e733cee32 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:50.202265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 560/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:45:50 2022] GET /identity/v3/users/f1b9757e13bb41eeadf9ce49e5e66356 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:50.208879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42854a70-97e5-4066-9470-3387a57038c2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:50.274720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:50.316290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42854a70-97e5-4066-9470-3387a57038c2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:50.352052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac778-9043-4a56-b61b-0b386335975f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:50.352717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac778-9043-4a56-b61b-0b386335975f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:50.352991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac778-9043-4a56-b61b-0b386335975f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:50.353264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac778-9043-4a56-b61b-0b386335975f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:50.420440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eac778-9043-4a56-b61b-0b386335975f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:50.421882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eac778-9043-4a56-b61b-0b386335975f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:50.487833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:50.531580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 568/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:45:50.537955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d3a18-4a7a-4404-af39-2a2e733cee32 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:50.602620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:50.655451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d3a18-4a7a-4404-af39-2a2e733cee32 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:50.701679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073eddc-78b7-48ab-b7e0-8a76b6954c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:50.702544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073eddc-78b7-48ab-b7e0-8a76b6954c45 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:50.702832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073eddc-78b7-48ab-b7e0-8a76b6954c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:50.703101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073eddc-78b7-48ab-b7e0-8a76b6954c45 None admin] PATH_INFO: `/v3/projects/a8458669d0a74badb671347fca61e535` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:50.704449 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073eddc-78b7-48ab-b7e0-8a76b6954c45 None admin] RBAC: Authorizing `identity:get_project(project_id=a8458669d0a74badb671347fca61e535)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:50.705895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073eddc-78b7-48ab-b7e0-8a76b6954c45 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:50.707836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 561/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:45:50 2022] GET /identity/v3/projects/a8458669d0a74badb671347fca61e535 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:50.717814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac778-9043-4a56-b61b-0b386335975f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:50.828841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac778-9043-4a56-b61b-0b386335975f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:50.866159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73bc32-17ab-4efc-a553-35fed5ebf40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:50.866809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73bc32-17ab-4efc-a553-35fed5ebf40c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:50.867087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73bc32-17ab-4efc-a553-35fed5ebf40c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:50.867357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73bc32-17ab-4efc-a553-35fed5ebf40c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:50.868583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73bc32-17ab-4efc-a553-35fed5ebf40c None admin] RBAC: Adding query filter params (role.id=e2c9006dc77f473f8e1064f285400a98, scope.project.id=a8458669d0a74badb671347fca61e535, user.id=f1b9757e13bb41eeadf9ce49e5e66356) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:50.868891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73bc32-17ab-4efc-a553-35fed5ebf40c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:50.870327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73bc32-17ab-4efc-a553-35fed5ebf40c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:50.876315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 569/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:50 2022] GET /identity/v3/role_assignments?user.id=f1b9757e13bb41eeadf9ce49e5e66356&scope.project.id=a8458669d0a74badb671347fca61e535&role.id=e2c9006dc77f473f8e1064f285400a98 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:52.944611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c34200-93cf-4bef-b173-e9b921e752d4 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:52.945205 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c34200-93cf-4bef-b173-e9b921e752d4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:52.945205 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c34200-93cf-4bef-b173-e9b921e752d4 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:52.946919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 562/1131] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:52.953343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008659c-7569-44e5-b08f-a2a23d7aaea0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:52.954104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008659c-7569-44e5-b08f-a2a23d7aaea0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:52.954237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008659c-7569-44e5-b08f-a2a23d7aaea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:52.974517 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2008659c-7569-44e5-b08f-a2a23d7aaea0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:52.975706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2008659c-7569-44e5-b08f-a2a23d7aaea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:52.994309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:53.049498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 570/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:53.079502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c34200-93cf-4bef-b173-e9b921e752d4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:53.190213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c34200-93cf-4bef-b173-e9b921e752d4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:53.214427 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:53.228053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed86a5-b74e-4e19-abdf-c8b427461691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:53.228906 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed86a5-b74e-4e19-abdf-c8b427461691 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:53.229184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed86a5-b74e-4e19-abdf-c8b427461691 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:53.229458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed86a5-b74e-4e19-abdf-c8b427461691 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:53.230785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed86a5-b74e-4e19-abdf-c8b427461691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:53.232279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed86a5-b74e-4e19-abdf-c8b427461691 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:53.233929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 563/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:53.240083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008659c-7569-44e5-b08f-a2a23d7aaea0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:53.347256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008659c-7569-44e5-b08f-a2a23d7aaea0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:53.399372 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce009f-bd4d-4a2c-b9f1-dfc88e5c2e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:53.400031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce009f-bd4d-4a2c-b9f1-dfc88e5c2e6e None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:53.400337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce009f-bd4d-4a2c-b9f1-dfc88e5c2e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:53.400606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce009f-bd4d-4a2c-b9f1-dfc88e5c2e6e None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:53.401314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce009f-bd4d-4a2c-b9f1-dfc88e5c2e6e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:53.402762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce009f-bd4d-4a2c-b9f1-dfc88e5c2e6e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:53.433716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 571/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:45:53 2022] POST /identity/v3/projects => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:55.484033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1da9c1-2dbb-4397-af5d-faff1fda3a3f None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:55.484606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1da9c1-2dbb-4397-af5d-faff1fda3a3f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:55.484606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1da9c1-2dbb-4397-af5d-faff1fda3a3f None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:55.486741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 564/1135] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:55.494736 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a318c-2a16-4041-b009-f3b92783210e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:55.495194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a318c-2a16-4041-b009-f3b92783210e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:55.495292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a318c-2a16-4041-b009-f3b92783210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:55.502378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:55.519812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-125a318c-2a16-4041-b009-f3b92783210e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:55.522267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-125a318c-2a16-4041-b009-f3b92783210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:55.594150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 572/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:45:55.619147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1da9c1-2dbb-4397-af5d-faff1fda3a3f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:55.728778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1da9c1-2dbb-4397-af5d-faff1fda3a3f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:55.763330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee6db-d110-42b1-9970-e65f7f0d00b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:55.764043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee6db-d110-42b1-9970-e65f7f0d00b4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:55.764348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee6db-d110-42b1-9970-e65f7f0d00b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:55.764616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee6db-d110-42b1-9970-e65f7f0d00b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:55.765810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ee6db-d110-42b1-9970-e65f7f0d00b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:55.767247 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ee6db-d110-42b1-9970-e65f7f0d00b4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:55.769011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 565/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:45:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:55.775397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a318c-2a16-4041-b009-f3b92783210e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:55.901741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a318c-2a16-4041-b009-f3b92783210e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:55.935960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae23e19-40f9-40a0-9c30-cb8d63ad1652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:55.936659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae23e19-40f9-40a0-9c30-cb8d63ad1652 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:55.936938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae23e19-40f9-40a0-9c30-cb8d63ad1652 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:55.937210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae23e19-40f9-40a0-9c30-cb8d63ad1652 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:55.937908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae23e19-40f9-40a0-9c30-cb8d63ad1652 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:55.939351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae23e19-40f9-40a0-9c30-cb8d63ad1652 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:55.946034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:55.964685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 573/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:45:55 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:45:58.009530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd1d8-73ef-4ed4-bcb1-359eb538ecf1 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:58.010103 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd1d8-73ef-4ed4-bcb1-359eb538ecf1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:58.010103 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd1d8-73ef-4ed4-bcb1-359eb538ecf1 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:58.011491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 566/1139] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:45:58.017881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf508130-d9c0-45bb-833c-53a36eb64e28 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:58.018674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf508130-d9c0-45bb-833c-53a36eb64e28 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:58.018945 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf508130-d9c0-45bb-833c-53a36eb64e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:58.038405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bf508130-d9c0-45bb-833c-53a36eb64e28 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:45:58.039585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bf508130-d9c0-45bb-833c-53a36eb64e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:45:58.116883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 574/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:45:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:45:58.140523 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd1d8-73ef-4ed4-bcb1-359eb538ecf1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:58.213192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:58.266202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd1d8-73ef-4ed4-bcb1-359eb538ecf1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:58.302990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d52820-04eb-4a52-8064-136496bf1f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:58.303790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d52820-04eb-4a52-8064-136496bf1f42 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:58.304077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d52820-04eb-4a52-8064-136496bf1f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:58.304382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d52820-04eb-4a52-8064-136496bf1f42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:58.316243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d52820-04eb-4a52-8064-136496bf1f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-90d52820-04eb-4a52-8064-136496bf1f42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:45:58.324222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:45:58.325343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:45:58.326847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 567/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:45:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:45:58.334215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf508130-d9c0-45bb-833c-53a36eb64e28 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:58.453494 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf508130-d9c0-45bb-833c-53a36eb64e28 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:58.475333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:58.489859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab6f56-fd2c-4a32-b444-ad1bc79b051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:58.490521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab6f56-fd2c-4a32-b444-ad1bc79b051d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:58.490796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab6f56-fd2c-4a32-b444-ad1bc79b051d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:58.491067 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab6f56-fd2c-4a32-b444-ad1bc79b051d None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:58.491822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab6f56-fd2c-4a32-b444-ad1bc79b051d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:58.492148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab6f56-fd2c-4a32-b444-ad1bc79b051d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:58.493525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab6f56-fd2c-4a32-b444-ad1bc79b051d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:58.501439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 575/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:45:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:58.507636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d52820-04eb-4a52-8064-136496bf1f42 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:58.617861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d52820-04eb-4a52-8064-136496bf1f42 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:58.634731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:58.652894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408f61-6bb1-4c83-8c6c-0e9cf1d092a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:58.653769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408f61-6bb1-4c83-8c6c-0e9cf1d092a2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:58.654046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408f61-6bb1-4c83-8c6c-0e9cf1d092a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:58.654315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408f61-6bb1-4c83-8c6c-0e9cf1d092a2 None admin] PATH_INFO: `/v3/users/acabc714204a4ffb88d450f82d4297f1` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:58.667893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a408f61-6bb1-4c83-8c6c-0e9cf1d092a2 None admin] RBAC: Authorizing `identity:get_user(user_id=acabc714204a4ffb88d450f82d4297f1)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:58.669383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a408f61-6bb1-4c83-8c6c-0e9cf1d092a2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:58.671038 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 568/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:45:58 2022] GET /identity/v3/users/acabc714204a4ffb88d450f82d4297f1 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:45:58.677098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab6f56-fd2c-4a32-b444-ad1bc79b051d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:58.778065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab6f56-fd2c-4a32-b444-ad1bc79b051d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:58.817920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d8d90-d4b3-42e2-be60-fb18769160ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:58.818736 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d8d90-d4b3-42e2-be60-fb18769160ef None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:58.819021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d8d90-d4b3-42e2-be60-fb18769160ef None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:58.819293 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d8d90-d4b3-42e2-be60-fb18769160ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:58.872551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:58.899889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d8d90-d4b3-42e2-be60-fb18769160ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:58.901386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d8d90-d4b3-42e2-be60-fb18769160ef None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:59.000599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 576/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:45:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:45:59.007688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408f61-6bb1-4c83-8c6c-0e9cf1d092a2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:59.117319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408f61-6bb1-4c83-8c6c-0e9cf1d092a2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:59.152637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d66115-80cb-412f-bb1d-3231c04aa64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:59.153490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d66115-80cb-412f-bb1d-3231c04aa64a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:59.153769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d66115-80cb-412f-bb1d-3231c04aa64a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:59.154041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d66115-80cb-412f-bb1d-3231c04aa64a None admin] PATH_INFO: `/v3/projects/784504cd90ea4b3a873eab58832121ef` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:59.155331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d66115-80cb-412f-bb1d-3231c04aa64a None admin] RBAC: Authorizing `identity:get_project(project_id=784504cd90ea4b3a873eab58832121ef)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:59.156787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d66115-80cb-412f-bb1d-3231c04aa64a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:59.158477 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 569/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:45:59 2022] GET /identity/v3/projects/784504cd90ea4b3a873eab58832121ef => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:45:59.164954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d8d90-d4b3-42e2-be60-fb18769160ef None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:45:59.252419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:45:59.273631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d8d90-d4b3-42e2-be60-fb18769160ef None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:45:59.309923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7475405-e84e-4b10-83bc-5efff005d52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:45:59.310563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7475405-e84e-4b10-83bc-5efff005d52d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:45:59.310838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7475405-e84e-4b10-83bc-5efff005d52d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:45:59.311108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7475405-e84e-4b10-83bc-5efff005d52d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:45:59.311912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7475405-e84e-4b10-83bc-5efff005d52d None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=784504cd90ea4b3a873eab58832121ef, user.id=acabc714204a4ffb88d450f82d4297f1) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:45:59.312269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7475405-e84e-4b10-83bc-5efff005d52d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:45:59.313675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7475405-e84e-4b10-83bc-5efff005d52d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:45:59.319391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 577/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:45:59 2022] GET /identity/v3/role_assignments?user.id=acabc714204a4ffb88d450f82d4297f1&scope.project.id=784504cd90ea4b3a873eab58832121ef&role.id=60bd1c44ce154338a7face8949d07281 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:01.283128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83565435-38b7-4c9f-b45a-0433ac6a0e99 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:01.283711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83565435-38b7-4c9f-b45a-0433ac6a0e99 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:01.283711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83565435-38b7-4c9f-b45a-0433ac6a0e99 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:01.285075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 570/1147] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:46:01.291478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b381f7-7071-4926-beca-a9df9bd321f1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:01.292229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b381f7-7071-4926-beca-a9df9bd321f1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:01.292399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b381f7-7071-4926-beca-a9df9bd321f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:01.319773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-24b381f7-7071-4926-beca-a9df9bd321f1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:46:01.321035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-24b381f7-7071-4926-beca-a9df9bd321f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:46:01.392577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 578/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:46:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:46:01.415023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83565435-38b7-4c9f-b45a-0433ac6a0e99 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:01.469149 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:01.527040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83565435-38b7-4c9f-b45a-0433ac6a0e99 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:01.562262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b2c1b-d75b-45e9-8328-751d3b2016ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:01.562985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b2c1b-d75b-45e9-8328-751d3b2016ec None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:01.563262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b2c1b-d75b-45e9-8328-751d3b2016ec None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:01.563531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b2c1b-d75b-45e9-8328-751d3b2016ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:01.574607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b2c1b-d75b-45e9-8328-751d3b2016ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d22b2c1b-d75b-45e9-8328-751d3b2016ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:46:01.584684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:46:01.585740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:46:01.586752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:46:01.588293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 571/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:46:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:46:01.592813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b381f7-7071-4926-beca-a9df9bd321f1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:01.676919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:01.714010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b381f7-7071-4926-beca-a9df9bd321f1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:01.749576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ca1ec-3ada-4207-877c-c119c6bd94be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:01.750236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ca1ec-3ada-4207-877c-c119c6bd94be None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:01.750514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ca1ec-3ada-4207-877c-c119c6bd94be None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:01.750792 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ca1ec-3ada-4207-877c-c119c6bd94be None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:01.751545 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ca1ec-3ada-4207-877c-c119c6bd94be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:46:01.751852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ca1ec-3ada-4207-877c-c119c6bd94be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:01.753263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ca1ec-3ada-4207-877c-c119c6bd94be None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:01.761634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 579/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:46:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:01.770975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b2c1b-d75b-45e9-8328-751d3b2016ec None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:01.891118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b2c1b-d75b-45e9-8328-751d3b2016ec None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:01.893526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:01.940546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d327d3-ca37-4a67-ba65-d008f09b6811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:01.941391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d327d3-ca37-4a67-ba65-d008f09b6811 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:01.941706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d327d3-ca37-4a67-ba65-d008f09b6811 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:01.941971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d327d3-ca37-4a67-ba65-d008f09b6811 None admin] PATH_INFO: `/v3/users/acabc714204a4ffb88d450f82d4297f1` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:01.943555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d327d3-ca37-4a67-ba65-d008f09b6811 None admin] RBAC: Authorizing `identity:get_user(user_id=acabc714204a4ffb88d450f82d4297f1)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:01.945024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d327d3-ca37-4a67-ba65-d008f09b6811 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:01.947034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 572/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:46:01 2022] GET /identity/v3/users/acabc714204a4ffb88d450f82d4297f1 => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:01.956795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ca1ec-3ada-4207-877c-c119c6bd94be None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:02.082005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ca1ec-3ada-4207-877c-c119c6bd94be None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:02.117760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7e6d1-efda-4d6d-8d70-358dce94e507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:02.119042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7e6d1-efda-4d6d-8d70-358dce94e507 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:02.119423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7e6d1-efda-4d6d-8d70-358dce94e507 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:02.119779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7e6d1-efda-4d6d-8d70-358dce94e507 None admin] PATH_INFO: `/v3/projects/784504cd90ea4b3a873eab58832121ef` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:02.121889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7e6d1-efda-4d6d-8d70-358dce94e507 None admin] RBAC: Authorizing `identity:get_project(project_id=784504cd90ea4b3a873eab58832121ef)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:02.124201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7e6d1-efda-4d6d-8d70-358dce94e507 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:02.126849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 580/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:46:01 2022] GET /identity/v3/projects/784504cd90ea4b3a873eab58832121ef => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:02.133317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d327d3-ca37-4a67-ba65-d008f09b6811 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:02.238009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d327d3-ca37-4a67-ba65-d008f09b6811 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:02.272845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1d125-e951-43a7-a5fd-e0fe4218acc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:02.273717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1d125-e951-43a7-a5fd-e0fe4218acc1 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:02.273990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1d125-e951-43a7-a5fd-e0fe4218acc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:02.274259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1d125-e951-43a7-a5fd-e0fe4218acc1 None admin] PATH_INFO: `/v3/projects/784504cd90ea4b3a873eab58832121ef/users/acabc714204a4ffb88d450f82d4297f1/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:02.276475 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1d125-e951-43a7-a5fd-e0fe4218acc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=784504cd90ea4b3a873eab58832121ef, user_id=acabc714204a4ffb88d450f82d4297f1, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:02.277886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1d125-e951-43a7-a5fd-e0fe4218acc1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:02.294722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 573/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:46:02 2022] PUT /identity/v3/projects/784504cd90ea4b3a873eab58832121ef/users/acabc714204a4ffb88d450f82d4297f1/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:46:04.270375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560350b9-baf9-4b4e-bd56-ec141041e627 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:04.271220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560350b9-baf9-4b4e-bd56-ec141041e627 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:04.271220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560350b9-baf9-4b4e-bd56-ec141041e627 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:04.273409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 581/1154] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:46:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:46:04.278822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300f6e5-e740-4e4e-a2a0-65d0b8ba0ecc None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:04.279253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300f6e5-e740-4e4e-a2a0-65d0b8ba0ecc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:04.279345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300f6e5-e740-4e4e-a2a0-65d0b8ba0ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:04.298128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8300f6e5-e740-4e4e-a2a0-65d0b8ba0ecc None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:46:04.299313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8300f6e5-e740-4e4e-a2a0-65d0b8ba0ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:46:04.328377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:04.380631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 574/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:46:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:46:04.420860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560350b9-baf9-4b4e-bd56-ec141041e627 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:04.436652 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:04.542406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560350b9-baf9-4b4e-bd56-ec141041e627 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:04.587432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e468b6c-f8fd-4b2e-a37f-4ee45edd8b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:04.588262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e468b6c-f8fd-4b2e-a37f-4ee45edd8b4a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:04.588548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e468b6c-f8fd-4b2e-a37f-4ee45edd8b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:04.588824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e468b6c-f8fd-4b2e-a37f-4ee45edd8b4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:04.600477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e468b6c-f8fd-4b2e-a37f-4ee45edd8b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5e468b6c-f8fd-4b2e-a37f-4ee45edd8b4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:46:04.608049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:46:04.609109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:46:04.610062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 582/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:46:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:46:04.616888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300f6e5-e740-4e4e-a2a0-65d0b8ba0ecc None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:04.720756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300f6e5-e740-4e4e-a2a0-65d0b8ba0ecc None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:04.758060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74900893-7bdb-478c-b806-9872716e6154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:04.758734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900893-7bdb-478c-b806-9872716e6154 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:04.759012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900893-7bdb-478c-b806-9872716e6154 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:04.759288 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900893-7bdb-478c-b806-9872716e6154 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:04.760051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900893-7bdb-478c-b806-9872716e6154 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:46:04.760388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900893-7bdb-478c-b806-9872716e6154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:04.761773 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900893-7bdb-478c-b806-9872716e6154 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:04.772739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 575/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:46:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:04.781219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e468b6c-f8fd-4b2e-a37f-4ee45edd8b4a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:04.855500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:04.892012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e468b6c-f8fd-4b2e-a37f-4ee45edd8b4a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:04.926914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528aa11-8a08-4427-8138-1529022824d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:04.927787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528aa11-8a08-4427-8138-1529022824d9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:04.928070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528aa11-8a08-4427-8138-1529022824d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:04.928365 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528aa11-8a08-4427-8138-1529022824d9 None admin] PATH_INFO: `/v3/users/acabc714204a4ffb88d450f82d4297f1` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:04.929584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528aa11-8a08-4427-8138-1529022824d9 None admin] RBAC: Authorizing `identity:get_user(user_id=acabc714204a4ffb88d450f82d4297f1)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:04.931018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528aa11-8a08-4427-8138-1529022824d9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:04.932682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 583/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:46:04 2022] GET /identity/v3/users/acabc714204a4ffb88d450f82d4297f1 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:46:04.939277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74900893-7bdb-478c-b806-9872716e6154 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:04.974575 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:05.056397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74900893-7bdb-478c-b806-9872716e6154 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:05.098109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eaa48f-fed8-4c33-876c-fddb556d020d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:05.098698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eaa48f-fed8-4c33-876c-fddb556d020d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:05.098973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eaa48f-fed8-4c33-876c-fddb556d020d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:05.099242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eaa48f-fed8-4c33-876c-fddb556d020d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:05.169340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eaa48f-fed8-4c33-876c-fddb556d020d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:05.170794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eaa48f-fed8-4c33-876c-fddb556d020d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:05.197351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:05.278974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 576/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:46:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:46:05.287511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528aa11-8a08-4427-8138-1529022824d9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:05.399721 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528aa11-8a08-4427-8138-1529022824d9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:05.434718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d2431-eb81-4695-86da-aa38d2f761e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:05.435573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d2431-eb81-4695-86da-aa38d2f761e5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:05.435853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d2431-eb81-4695-86da-aa38d2f761e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:05.436150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d2431-eb81-4695-86da-aa38d2f761e5 None admin] PATH_INFO: `/v3/projects/784504cd90ea4b3a873eab58832121ef` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:05.437360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d2431-eb81-4695-86da-aa38d2f761e5 None admin] RBAC: Authorizing `identity:get_project(project_id=784504cd90ea4b3a873eab58832121ef)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:05.438791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d2431-eb81-4695-86da-aa38d2f761e5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:05.440508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 584/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:46:05 2022] GET /identity/v3/projects/784504cd90ea4b3a873eab58832121ef => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:05.450879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eaa48f-fed8-4c33-876c-fddb556d020d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:05.562040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eaa48f-fed8-4c33-876c-fddb556d020d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:05.587118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:05.599565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d39073-eda7-4a97-b89f-2d3b1874a1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:05.600248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d39073-eda7-4a97-b89f-2d3b1874a1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:05.600529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d39073-eda7-4a97-b89f-2d3b1874a1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:05.600801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d39073-eda7-4a97-b89f-2d3b1874a1f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:05.601648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d39073-eda7-4a97-b89f-2d3b1874a1f4 None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=784504cd90ea4b3a873eab58832121ef, user.id=acabc714204a4ffb88d450f82d4297f1) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:46:05.601953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d39073-eda7-4a97-b89f-2d3b1874a1f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:05.603397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d39073-eda7-4a97-b89f-2d3b1874a1f4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:05.609426 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 577/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:46:05 2022] GET /identity/v3/role_assignments?user.id=acabc714204a4ffb88d450f82d4297f1&scope.project.id=784504cd90ea4b3a873eab58832121ef&role.id=60bd1c44ce154338a7face8949d07281 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:07.694711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545c430-b18e-4ef0-90e1-6a8ae2601bdf None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:07.695566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545c430-b18e-4ef0-90e1-6a8ae2601bdf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:07.695566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545c430-b18e-4ef0-90e1-6a8ae2601bdf None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:07.697493 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 585/1162] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:46:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:46:07.703771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a4b80-8ca1-4abe-b3dc-6832c75f9a70 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:07.704520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a4b80-8ca1-4abe-b3dc-6832c75f9a70 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:07.704696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a4b80-8ca1-4abe-b3dc-6832c75f9a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:07.725661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d32a4b80-8ca1-4abe-b3dc-6832c75f9a70 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:46:07.726849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d32a4b80-8ca1-4abe-b3dc-6832c75f9a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:46:07.798535 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 578/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:46:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:46:07.825702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545c430-b18e-4ef0-90e1-6a8ae2601bdf None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:07.829672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:07.950809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545c430-b18e-4ef0-90e1-6a8ae2601bdf None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:07.985957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95dd51-a421-4a16-a90f-450e80d98ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:07.986598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95dd51-a421-4a16-a90f-450e80d98ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:07.986873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95dd51-a421-4a16-a90f-450e80d98ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:07.987142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95dd51-a421-4a16-a90f-450e80d98ab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:08.055819 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95dd51-a421-4a16-a90f-450e80d98ab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:08.057325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95dd51-a421-4a16-a90f-450e80d98ab7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:08.059155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:08.164187 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 586/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:46:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:46:08.173749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a4b80-8ca1-4abe-b3dc-6832c75f9a70 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:08.289719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a4b80-8ca1-4abe-b3dc-6832c75f9a70 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:08.307298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:08.325297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2e9d8-ceed-43a6-b322-d4ad000af8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:08.326106 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2e9d8-ceed-43a6-b322-d4ad000af8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:08.326388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2e9d8-ceed-43a6-b322-d4ad000af8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:08.326640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2e9d8-ceed-43a6-b322-d4ad000af8e1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:08.333591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2e9d8-ceed-43a6-b322-d4ad000af8e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:08.335046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2e9d8-ceed-43a6-b322-d4ad000af8e1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-79a2e9d8-ceed-43a6-b322-d4ad000af8e1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:46:08.350105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:46:08.351241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:46:08.352225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 579/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jul 11 20:46:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:46:08.358162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95dd51-a421-4a16-a90f-450e80d98ab7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:08.467762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95dd51-a421-4a16-a90f-450e80d98ab7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:08.469570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:08.504936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0f454-9a8e-4fae-9e9c-694bd9789cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:08.505605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0f454-9a8e-4fae-9e9c-694bd9789cfc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:08.505884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0f454-9a8e-4fae-9e9c-694bd9789cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:08.506157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0f454-9a8e-4fae-9e9c-694bd9789cfc None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:08.506892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0f454-9a8e-4fae-9e9c-694bd9789cfc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:46:08.507198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0f454-9a8e-4fae-9e9c-694bd9789cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:08.508622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0f454-9a8e-4fae-9e9c-694bd9789cfc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:08.515887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 587/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jul 11 20:46:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:10.567426 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccbcdb-80ff-4b31-9a45-198a91fdf3fa None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:10.568294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccbcdb-80ff-4b31-9a45-198a91fdf3fa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:10.568460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccbcdb-80ff-4b31-9a45-198a91fdf3fa None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:10.570694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 580/1167] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:46:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:46:10.577390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c27cf6-949d-4cd4-9968-37706ab14453 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:10.577813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c27cf6-949d-4cd4-9968-37706ab14453 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:10.577993 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c27cf6-949d-4cd4-9968-37706ab14453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:10.597268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c3c27cf6-949d-4cd4-9968-37706ab14453 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:46:10.598456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c3c27cf6-949d-4cd4-9968-37706ab14453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:46:10.687202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 588/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:46:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:46:10.711226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccbcdb-80ff-4b31-9a45-198a91fdf3fa None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:10.822100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccbcdb-80ff-4b31-9a45-198a91fdf3fa None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:10.864929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6d374-ecd0-473a-bb14-00dc03e5535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:10.865602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6d374-ecd0-473a-bb14-00dc03e5535e None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:10.865881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6d374-ecd0-473a-bb14-00dc03e5535e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:10.866158 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6d374-ecd0-473a-bb14-00dc03e5535e None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:10.866918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6d374-ecd0-473a-bb14-00dc03e5535e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:10.868341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6d374-ecd0-473a-bb14-00dc03e5535e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:10.891563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 581/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Mon Jul 11 20:46:10 2022] POST /identity/v3/domains => generated 267 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:46:12.969583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218edf9-d168-4c62-9ea4-70a69554d81c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:12.970120 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218edf9-d168-4c62-9ea4-70a69554d81c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:12.970120 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218edf9-d168-4c62-9ea4-70a69554d81c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:12.971525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 589/1170] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:46:12.978139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e38cf-60b4-4c92-9f72-8e552e23ca86 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:12.978805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e38cf-60b4-4c92-9f72-8e552e23ca86 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:12.978995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e38cf-60b4-4c92-9f72-8e552e23ca86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:13.000265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a51e38cf-60b4-4c92-9f72-8e552e23ca86 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:46:13.001438 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a51e38cf-60b4-4c92-9f72-8e552e23ca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:46:13.032818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:13.078330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 582/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:46:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:46:13.101961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218edf9-d168-4c62-9ea4-70a69554d81c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:13.204298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:13.212547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218edf9-d168-4c62-9ea4-70a69554d81c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:13.248956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2238778-2d92-4aac-881c-0a15cf8d1df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:13.249802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2238778-2d92-4aac-881c-0a15cf8d1df8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:13.250079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2238778-2d92-4aac-881c-0a15cf8d1df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:13.250349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2238778-2d92-4aac-881c-0a15cf8d1df8 None admin] PATH_INFO: `/v3/domains/61af5e50937e42e68287c57e01de5d19` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:13.261527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2238778-2d92-4aac-881c-0a15cf8d1df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=61af5e50937e42e68287c57e01de5d19)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:13.262982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2238778-2d92-4aac-881c-0a15cf8d1df8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:13.264624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 590/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Jul 11 20:46:13 2022] GET /identity/v3/domains/61af5e50937e42e68287c57e01de5d19 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:13.271144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e38cf-60b4-4c92-9f72-8e552e23ca86 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:13.383454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e38cf-60b4-4c92-9f72-8e552e23ca86 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:13.421829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b15492-63de-422e-84e4-72ed9a8a14b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:13.422473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b15492-63de-422e-84e4-72ed9a8a14b0 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:13.422753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b15492-63de-422e-84e4-72ed9a8a14b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:13.423018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b15492-63de-422e-84e4-72ed9a8a14b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:13.423726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b15492-63de-422e-84e4-72ed9a8a14b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:13.425188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b15492-63de-422e-84e4-72ed9a8a14b0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:13.454352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 583/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:46:13 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:46:15.466696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e211-961a-4815-8269-0a0b6eadf561 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:15.467283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e211-961a-4815-8269-0a0b6eadf561 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:15.467283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e211-961a-4815-8269-0a0b6eadf561 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:15.468650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 591/1174] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:46:15.475226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6730a9-d574-4a34-a5a8-e682e998fb9f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:15.475887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6730a9-d574-4a34-a5a8-e682e998fb9f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:15.476089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6730a9-d574-4a34-a5a8-e682e998fb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:15.504393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0b6730a9-d574-4a34-a5a8-e682e998fb9f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:46:15.505586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0b6730a9-d574-4a34-a5a8-e682e998fb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:46:15.521538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:15.577982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 584/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:46:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:46:15.610159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5e211-961a-4815-8269-0a0b6eadf561 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:15.726472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5e211-961a-4815-8269-0a0b6eadf561 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:15.775720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd55c9-af93-4384-962c-764029ea2a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:15.776646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd55c9-af93-4384-962c-764029ea2a8b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:15.777082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd55c9-af93-4384-962c-764029ea2a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:15.777520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd55c9-af93-4384-962c-764029ea2a8b None admin] PATH_INFO: `/v3/domains/61af5e50937e42e68287c57e01de5d19` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:15.779528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd55c9-af93-4384-962c-764029ea2a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=61af5e50937e42e68287c57e01de5d19)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:15.781119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd55c9-af93-4384-962c-764029ea2a8b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:15.783504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 592/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Jul 11 20:46:15 2022] GET /identity/v3/domains/61af5e50937e42e68287c57e01de5d19 => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:15.790258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6730a9-d574-4a34-a5a8-e682e998fb9f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:15.903326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6730a9-d574-4a34-a5a8-e682e998fb9f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:15.938423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a241c08-b317-44c1-b0b2-b89b43790969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:15.939092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a241c08-b317-44c1-b0b2-b89b43790969 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:15.939370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a241c08-b317-44c1-b0b2-b89b43790969 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:15.939642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a241c08-b317-44c1-b0b2-b89b43790969 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:15.940394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a241c08-b317-44c1-b0b2-b89b43790969 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:15.941856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a241c08-b317-44c1-b0b2-b89b43790969 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:15.965195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 585/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:46:15 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:46:18.005790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3a25-006f-4b29-a36f-29fcb0425503 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:18.006382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3a25-006f-4b29-a36f-29fcb0425503 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:18.006382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3a25-006f-4b29-a36f-29fcb0425503 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:18.007706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 593/1178] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:46:18.014096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55df7-1d0f-49da-aaac-86b30e786b28 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:18.014807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55df7-1d0f-49da-aaac-86b30e786b28 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:18.015003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55df7-1d0f-49da-aaac-86b30e786b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:18.034219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:18.040805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3ad55df7-1d0f-49da-aaac-86b30e786b28 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:46:18.042543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3ad55df7-1d0f-49da-aaac-86b30e786b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:46:18.125921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 586/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:46:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:46:18.156586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a3a25-006f-4b29-a36f-29fcb0425503 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:18.183773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:18.263738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a3a25-006f-4b29-a36f-29fcb0425503 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:18.300925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5809c-d66a-4a32-8c0f-ec5208c0bf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:18.301718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5809c-d66a-4a32-8c0f-ec5208c0bf8e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:18.301994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5809c-d66a-4a32-8c0f-ec5208c0bf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:18.302265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5809c-d66a-4a32-8c0f-ec5208c0bf8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:18.313550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5809c-d66a-4a32-8c0f-ec5208c0bf8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-61f5809c-d66a-4a32-8c0f-ec5208c0bf8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:46:18.321555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:46:18.322690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:46:18.323742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:46:18.327639 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 594/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:46:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:46:18.328785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55df7-1d0f-49da-aaac-86b30e786b28 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:18.433601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55df7-1d0f-49da-aaac-86b30e786b28 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:18.475619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fb9db-975b-4fd9-8320-b1fcd859d250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:18.476323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fb9db-975b-4fd9-8320-b1fcd859d250 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:18.476606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fb9db-975b-4fd9-8320-b1fcd859d250 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:18.476881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fb9db-975b-4fd9-8320-b1fcd859d250 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:18.477697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fb9db-975b-4fd9-8320-b1fcd859d250 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:46:18.478009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fb9db-975b-4fd9-8320-b1fcd859d250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:18.479396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fb9db-975b-4fd9-8320-b1fcd859d250 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:18.481632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:18.489069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 587/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:46:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:46:18.495206 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5809c-d66a-4a32-8c0f-ec5208c0bf8e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:18.591450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:18.607349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5809c-d66a-4a32-8c0f-ec5208c0bf8e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:18.641853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8921b-a4f8-43f4-aac7-2efa8348538d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:18.642722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8921b-a4f8-43f4-aac7-2efa8348538d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:18.643001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8921b-a4f8-43f4-aac7-2efa8348538d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:18.643278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8921b-a4f8-43f4-aac7-2efa8348538d None admin] PATH_INFO: `/v3/users/904852654da54ddbbd4b25de155f0873` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:18.656458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8921b-a4f8-43f4-aac7-2efa8348538d None admin] RBAC: Authorizing `identity:get_user(user_id=904852654da54ddbbd4b25de155f0873)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:18.657896 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8921b-a4f8-43f4-aac7-2efa8348538d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:18.659514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 595/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:46:18 2022] GET /identity/v3/users/904852654da54ddbbd4b25de155f0873 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:18.668890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fb9db-975b-4fd9-8320-b1fcd859d250 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:18.778259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fb9db-975b-4fd9-8320-b1fcd859d250 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:18.812764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab8956-6eac-416a-87cd-9d5e271ecf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:18.813406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab8956-6eac-416a-87cd-9d5e271ecf3c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:18.813683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab8956-6eac-416a-87cd-9d5e271ecf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:18.813955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab8956-6eac-416a-87cd-9d5e271ecf3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:18.878273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:18.882221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab8956-6eac-416a-87cd-9d5e271ecf3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:18.883729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab8956-6eac-416a-87cd-9d5e271ecf3c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:18.992189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 588/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:46:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:46:18.998440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8921b-a4f8-43f4-aac7-2efa8348538d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:19.107911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8921b-a4f8-43f4-aac7-2efa8348538d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:19.144439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcba13e-3bdd-4d05-b1e9-00d80d052670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:19.145701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcba13e-3bdd-4d05-b1e9-00d80d052670 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:19.145982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcba13e-3bdd-4d05-b1e9-00d80d052670 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:19.146257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcba13e-3bdd-4d05-b1e9-00d80d052670 None admin] PATH_INFO: `/v3/projects/ca392bd6bf9e4038bfd4ae8fbc308033` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:19.147492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcba13e-3bdd-4d05-b1e9-00d80d052670 None admin] RBAC: Authorizing `identity:get_project(project_id=ca392bd6bf9e4038bfd4ae8fbc308033)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:19.148980 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcba13e-3bdd-4d05-b1e9-00d80d052670 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:19.150695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 596/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:46:18 2022] GET /identity/v3/projects/ca392bd6bf9e4038bfd4ae8fbc308033 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:19.157153 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab8956-6eac-416a-87cd-9d5e271ecf3c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:19.277860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:19.281925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab8956-6eac-416a-87cd-9d5e271ecf3c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:19.318538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546eefe3-6fdc-4db2-a1d9-b9b38bff0c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:19.319203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546eefe3-6fdc-4db2-a1d9-b9b38bff0c22 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:19.319480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546eefe3-6fdc-4db2-a1d9-b9b38bff0c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:19.319752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546eefe3-6fdc-4db2-a1d9-b9b38bff0c22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:19.320622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546eefe3-6fdc-4db2-a1d9-b9b38bff0c22 None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=ca392bd6bf9e4038bfd4ae8fbc308033, user.id=904852654da54ddbbd4b25de155f0873) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:46:19.321277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546eefe3-6fdc-4db2-a1d9-b9b38bff0c22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:19.322719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546eefe3-6fdc-4db2-a1d9-b9b38bff0c22 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:19.328709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 589/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:46:19 2022] GET /identity/v3/role_assignments?user.id=904852654da54ddbbd4b25de155f0873&scope.project.id=ca392bd6bf9e4038bfd4ae8fbc308033&role.id=60bd1c44ce154338a7face8949d07281 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:21.292880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d06e6-9dad-44c5-914e-e0500afb3c33 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:21.293773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d06e6-9dad-44c5-914e-e0500afb3c33 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:21.293773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d06e6-9dad-44c5-914e-e0500afb3c33 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:21.295884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 597/1186] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:46:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:46:21.304291 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6608-948f-4075-a48b-48c01b255e6c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:21.304734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6608-948f-4075-a48b-48c01b255e6c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:21.304858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6608-948f-4075-a48b-48c01b255e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:21.332979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0fdb6608-948f-4075-a48b-48c01b255e6c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:46:21.334180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0fdb6608-948f-4075-a48b-48c01b255e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:46:21.415731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 590/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:46:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:46:21.444711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d06e6-9dad-44c5-914e-e0500afb3c33 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:21.462356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:21.556951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d06e6-9dad-44c5-914e-e0500afb3c33 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:21.593319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96473813-d248-4241-b1ca-56360591c00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:21.594096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96473813-d248-4241-b1ca-56360591c00b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:21.594372 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96473813-d248-4241-b1ca-56360591c00b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:21.594643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96473813-d248-4241-b1ca-56360591c00b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:21.605405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96473813-d248-4241-b1ca-56360591c00b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-96473813-d248-4241-b1ca-56360591c00b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:46:21.612864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:46:21.614000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:46:21.614877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 598/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:46:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:46:21.622925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb6608-948f-4075-a48b-48c01b255e6c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:21.724018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:21.734105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb6608-948f-4075-a48b-48c01b255e6c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:21.769875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a750870-2a07-4a8d-b409-a715fc2f6af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:21.770610 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a750870-2a07-4a8d-b409-a715fc2f6af0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:21.770885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a750870-2a07-4a8d-b409-a715fc2f6af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:21.771160 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a750870-2a07-4a8d-b409-a715fc2f6af0 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:21.771911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a750870-2a07-4a8d-b409-a715fc2f6af0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:46:21.772222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a750870-2a07-4a8d-b409-a715fc2f6af0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:21.773622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a750870-2a07-4a8d-b409-a715fc2f6af0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:21.781865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 591/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:46:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:21.788708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96473813-d248-4241-b1ca-56360591c00b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:21.873668 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:21.905935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96473813-d248-4241-b1ca-56360591c00b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:21.939900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca415d45-7328-4d1f-96d2-65c6a5394e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:21.940791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca415d45-7328-4d1f-96d2-65c6a5394e1b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:21.941064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca415d45-7328-4d1f-96d2-65c6a5394e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:21.941346 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca415d45-7328-4d1f-96d2-65c6a5394e1b None admin] PATH_INFO: `/v3/users/904852654da54ddbbd4b25de155f0873` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:21.942637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca415d45-7328-4d1f-96d2-65c6a5394e1b None admin] RBAC: Authorizing `identity:get_user(user_id=904852654da54ddbbd4b25de155f0873)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:21.944070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca415d45-7328-4d1f-96d2-65c6a5394e1b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:21.946068 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 599/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:46:21 2022] GET /identity/v3/users/904852654da54ddbbd4b25de155f0873 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:21.952288 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a750870-2a07-4a8d-b409-a715fc2f6af0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:22.071055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a750870-2a07-4a8d-b409-a715fc2f6af0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:22.108123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db252c-7172-4c13-b050-fae0b7b32b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:22.109010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db252c-7172-4c13-b050-fae0b7b32b2d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:22.109285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db252c-7172-4c13-b050-fae0b7b32b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:22.109560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db252c-7172-4c13-b050-fae0b7b32b2d None admin] PATH_INFO: `/v3/projects/ca392bd6bf9e4038bfd4ae8fbc308033` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:22.110797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db252c-7172-4c13-b050-fae0b7b32b2d None admin] RBAC: Authorizing `identity:get_project(project_id=ca392bd6bf9e4038bfd4ae8fbc308033)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:22.112274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db252c-7172-4c13-b050-fae0b7b32b2d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:22.114331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 592/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:46:21 2022] GET /identity/v3/projects/ca392bd6bf9e4038bfd4ae8fbc308033 => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:22.120908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca415d45-7328-4d1f-96d2-65c6a5394e1b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:22.232426 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca415d45-7328-4d1f-96d2-65c6a5394e1b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:22.267150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fc1ba-6906-43f0-b266-099ae6110715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:22.268029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fc1ba-6906-43f0-b266-099ae6110715 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:22.268336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fc1ba-6906-43f0-b266-099ae6110715 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:22.268610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fc1ba-6906-43f0-b266-099ae6110715 None admin] PATH_INFO: `/v3/projects/ca392bd6bf9e4038bfd4ae8fbc308033/users/904852654da54ddbbd4b25de155f0873/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:22.270754 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fc1ba-6906-43f0-b266-099ae6110715 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca392bd6bf9e4038bfd4ae8fbc308033, user_id=904852654da54ddbbd4b25de155f0873, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:22.272207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fc1ba-6906-43f0-b266-099ae6110715 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:22.283771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:22.289513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 600/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:46:22 2022] PUT /identity/v3/projects/ca392bd6bf9e4038bfd4ae8fbc308033/users/904852654da54ddbbd4b25de155f0873/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:46:24.271438 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc08e49-8a17-44bd-8daa-6414ad65d206 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:24.272397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc08e49-8a17-44bd-8daa-6414ad65d206 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:24.272397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc08e49-8a17-44bd-8daa-6414ad65d206 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:24.274582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 593/1193] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:46:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:46:24.280986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d547f-0490-48ec-a6e2-90730c6ae69f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:24.281409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d547f-0490-48ec-a6e2-90730c6ae69f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:24.281550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d547f-0490-48ec-a6e2-90730c6ae69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:24.300493 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a66d547f-0490-48ec-a6e2-90730c6ae69f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:46:24.301723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a66d547f-0490-48ec-a6e2-90730c6ae69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:46:24.372895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 601/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:46:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:46:24.396765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc08e49-8a17-44bd-8daa-6414ad65d206 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:24.423787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:24.521687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc08e49-8a17-44bd-8daa-6414ad65d206 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:24.558975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a89d70-225a-41bd-b3ee-66245e7a54ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:24.559767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a89d70-225a-41bd-b3ee-66245e7a54ea None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:24.560049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a89d70-225a-41bd-b3ee-66245e7a54ea None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:24.560351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a89d70-225a-41bd-b3ee-66245e7a54ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:24.571863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a89d70-225a-41bd-b3ee-66245e7a54ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-25a89d70-225a-41bd-b3ee-66245e7a54ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:46:24.579488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:46:24.580624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:46:24.581640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 594/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:46:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:46:24.587082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d547f-0490-48ec-a6e2-90730c6ae69f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:24.697112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d547f-0490-48ec-a6e2-90730c6ae69f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:24.724032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:24.737128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1ca3-4a3b-4397-aa16-97104257943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:24.738090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1ca3-4a3b-4397-aa16-97104257943c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:24.738588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1ca3-4a3b-4397-aa16-97104257943c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:24.738865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1ca3-4a3b-4397-aa16-97104257943c None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:24.739623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e1ca3-4a3b-4397-aa16-97104257943c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:46:24.739933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e1ca3-4a3b-4397-aa16-97104257943c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:24.741353 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e1ca3-4a3b-4397-aa16-97104257943c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:24.750051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 602/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:46:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:24.756013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a89d70-225a-41bd-b3ee-66245e7a54ea None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:24.842890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:24.864403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a89d70-225a-41bd-b3ee-66245e7a54ea None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:24.905287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f076597-8f59-49f7-b6f7-fcec7966d361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:24.906163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f076597-8f59-49f7-b6f7-fcec7966d361 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:24.906411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f076597-8f59-49f7-b6f7-fcec7966d361 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:24.906680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f076597-8f59-49f7-b6f7-fcec7966d361 None admin] PATH_INFO: `/v3/users/904852654da54ddbbd4b25de155f0873` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:24.907892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f076597-8f59-49f7-b6f7-fcec7966d361 None admin] RBAC: Authorizing `identity:get_user(user_id=904852654da54ddbbd4b25de155f0873)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:24.909395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f076597-8f59-49f7-b6f7-fcec7966d361 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:24.911113 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 595/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jul 11 20:46:24 2022] GET /identity/v3/users/904852654da54ddbbd4b25de155f0873 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:24.917786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1ca3-4a3b-4397-aa16-97104257943c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:25.022685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1ca3-4a3b-4397-aa16-97104257943c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:25.059901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d38e1-4a39-4723-8c09-afecf84f1ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:25.060571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d38e1-4a39-4723-8c09-afecf84f1ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:25.060845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d38e1-4a39-4723-8c09-afecf84f1ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:25.061116 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d38e1-4a39-4723-8c09-afecf84f1ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:25.121497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:25.132495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d38e1-4a39-4723-8c09-afecf84f1ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:25.133933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d38e1-4a39-4723-8c09-afecf84f1ee0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:25.244685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 603/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:46:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 20:46:25.251613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f076597-8f59-49f7-b6f7-fcec7966d361 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:25.364813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f076597-8f59-49f7-b6f7-fcec7966d361 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:25.400710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd22c3-d0c1-4b15-b1eb-8c89c51c5e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:25.401497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd22c3-d0c1-4b15-b1eb-8c89c51c5e9c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:25.401767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd22c3-d0c1-4b15-b1eb-8c89c51c5e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:25.402031 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd22c3-d0c1-4b15-b1eb-8c89c51c5e9c None admin] PATH_INFO: `/v3/projects/ca392bd6bf9e4038bfd4ae8fbc308033` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:25.403336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd22c3-d0c1-4b15-b1eb-8c89c51c5e9c None admin] RBAC: Authorizing `identity:get_project(project_id=ca392bd6bf9e4038bfd4ae8fbc308033)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:25.404796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd22c3-d0c1-4b15-b1eb-8c89c51c5e9c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:25.406512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 596/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jul 11 20:46:25 2022] GET /identity/v3/projects/ca392bd6bf9e4038bfd4ae8fbc308033 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:25.413138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d38e1-4a39-4723-8c09-afecf84f1ee0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:46:25.504865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:25.515697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d38e1-4a39-4723-8c09-afecf84f1ee0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:46:25.554207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be775b-23d2-4498-9976-7f919f0505b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:46:25.554870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be775b-23d2-4498-9976-7f919f0505b8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:46:25.555147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be775b-23d2-4498-9976-7f919f0505b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:46:25.555419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be775b-23d2-4498-9976-7f919f0505b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:46:25.556283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be775b-23d2-4498-9976-7f919f0505b8 None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=ca392bd6bf9e4038bfd4ae8fbc308033, user.id=904852654da54ddbbd4b25de155f0873) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:46:25.556592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be775b-23d2-4498-9976-7f919f0505b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:46:25.558036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be775b-23d2-4498-9976-7f919f0505b8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:46:25.564053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 604/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:46:25 2022] GET /identity/v3/role_assignments?user.id=904852654da54ddbbd4b25de155f0873&scope.project.id=ca392bd6bf9e4038bfd4ae8fbc308033&role.id=60bd1c44ce154338a7face8949d07281 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:46:35.395808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:46:35.561420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=23 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:21.325993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032b9e8-3c89-46c3-a0c1-10719494306f None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:21.326584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032b9e8-3c89-46c3-a0c1-10719494306f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:21.326584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032b9e8-3c89-46c3-a0c1-10719494306f None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:21.328318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 597/1201] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:21.333829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82722da8-ef09-458b-8188-615534c506bf None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:21.334305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82722da8-ef09-458b-8188-615534c506bf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:21.334436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82722da8-ef09-458b-8188-615534c506bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:21.355750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-82722da8-ef09-458b-8188-615534c506bf None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:21.357444 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-82722da8-ef09-458b-8188-615534c506bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:21.438136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 605/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:47:21.466988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032b9e8-3c89-46c3-a0c1-10719494306f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:21.597183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032b9e8-3c89-46c3-a0c1-10719494306f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:21.638508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bf8b2-c7d6-48df-a23f-57542e842ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:21.639328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bf8b2-c7d6-48df-a23f-57542e842ded None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:21.639611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bf8b2-c7d6-48df-a23f-57542e842ded None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:21.639890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bf8b2-c7d6-48df-a23f-57542e842ded None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:21.652772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bf8b2-c7d6-48df-a23f-57542e842ded None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:21.654265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bf8b2-c7d6-48df-a23f-57542e842ded None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-142bf8b2-c7d6-48df-a23f-57542e842ded None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:21.674614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:21.675800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:21.677187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:21.677187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:21.677187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:21.677187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:21.677187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 598/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:21.684408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82722da8-ef09-458b-8188-615534c506bf None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:21.818530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82722da8-ef09-458b-8188-615534c506bf None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:21.862702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:21.871264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c51b5b-94c9-4906-83ac-c67b455d6559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:21.872178 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c51b5b-94c9-4906-83ac-c67b455d6559 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:21.872513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c51b5b-94c9-4906-83ac-c67b455d6559 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:21.872934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c51b5b-94c9-4906-83ac-c67b455d6559 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:21.873801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c51b5b-94c9-4906-83ac-c67b455d6559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:21.874124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c51b5b-94c9-4906-83ac-c67b455d6559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:21.875794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c51b5b-94c9-4906-83ac-c67b455d6559 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:21.889456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 606/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:21.895543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bf8b2-c7d6-48df-a23f-57542e842ded None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:21.961591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:22.022063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bf8b2-c7d6-48df-a23f-57542e842ded None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:22.076952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80836987-b39f-4751-b0d6-6987651af583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:22.077671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80836987-b39f-4751-b0d6-6987651af583 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:22.077956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80836987-b39f-4751-b0d6-6987651af583 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:22.078235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80836987-b39f-4751-b0d6-6987651af583 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:22.078976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80836987-b39f-4751-b0d6-6987651af583 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:22.080467 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80836987-b39f-4751-b0d6-6987651af583 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:22.104574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 599/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jul 11 20:47:21 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:47:24.236237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d86b54-fea8-4c25-87fa-6f70920694c3 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:24.236780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d86b54-fea8-4c25-87fa-6f70920694c3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:24.236888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d86b54-fea8-4c25-87fa-6f70920694c3 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:24.240186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 607/1206] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:24.244215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355381a-6554-4808-a239-b56a7692e689 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:24.244695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355381a-6554-4808-a239-b56a7692e689 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:24.244796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355381a-6554-4808-a239-b56a7692e689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:24.262288 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3355381a-6554-4808-a239-b56a7692e689 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:24.263504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3355381a-6554-4808-a239-b56a7692e689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:24.338381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 600/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:47:24.373032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d86b54-fea8-4c25-87fa-6f70920694c3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:24.495733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d86b54-fea8-4c25-87fa-6f70920694c3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:24.529672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff531d3-5067-48eb-9b24-4308c7a28967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:24.530502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff531d3-5067-48eb-9b24-4308c7a28967 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:24.530779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff531d3-5067-48eb-9b24-4308c7a28967 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:24.531056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff531d3-5067-48eb-9b24-4308c7a28967 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:24.542831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff531d3-5067-48eb-9b24-4308c7a28967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-3ff531d3-5067-48eb-9b24-4308c7a28967 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:47:24.550723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:24.552099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:24.552967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 608/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:47:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:24.558282 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355381a-6554-4808-a239-b56a7692e689 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:24.608347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:24.676353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355381a-6554-4808-a239-b56a7692e689 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:24.714292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954fb8f-2c66-4f9d-a43a-ac1ee3edad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:24.714986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954fb8f-2c66-4f9d-a43a-ac1ee3edad8b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:24.715274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954fb8f-2c66-4f9d-a43a-ac1ee3edad8b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:24.715544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954fb8f-2c66-4f9d-a43a-ac1ee3edad8b None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:24.716345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954fb8f-2c66-4f9d-a43a-ac1ee3edad8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:24.716659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954fb8f-2c66-4f9d-a43a-ac1ee3edad8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:24.718043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954fb8f-2c66-4f9d-a43a-ac1ee3edad8b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:24.727515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 601/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:47:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:24.733264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff531d3-5067-48eb-9b24-4308c7a28967 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:24.790629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:24.844999 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff531d3-5067-48eb-9b24-4308c7a28967 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:24.883323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb725798-ba51-488e-90d1-4925c474d7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:24.884130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb725798-ba51-488e-90d1-4925c474d7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:24.884430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb725798-ba51-488e-90d1-4925c474d7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:24.884708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb725798-ba51-488e-90d1-4925c474d7f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:24.895604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb725798-ba51-488e-90d1-4925c474d7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:24.897097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb725798-ba51-488e-90d1-4925c474d7f7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-cb725798-ba51-488e-90d1-4925c474d7f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:24.917538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:24.919542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:24.920870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:24.920870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:24.920870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:24.920870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:24.920870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 609/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:24.925774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954fb8f-2c66-4f9d-a43a-ac1ee3edad8b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:25.034924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:25.037888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954fb8f-2c66-4f9d-a43a-ac1ee3edad8b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:25.078356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c956a3-9b43-40f2-9242-69353c003d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:25.078993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c956a3-9b43-40f2-9242-69353c003d49 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:25.079270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c956a3-9b43-40f2-9242-69353c003d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:25.079562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c956a3-9b43-40f2-9242-69353c003d49 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:25.080319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c956a3-9b43-40f2-9242-69353c003d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:25.080626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c956a3-9b43-40f2-9242-69353c003d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:25.082069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c956a3-9b43-40f2-9242-69353c003d49 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:25.099537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 602/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:25.100858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb725798-ba51-488e-90d1-4925c474d7f7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:25.218193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb725798-ba51-488e-90d1-4925c474d7f7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:25.220064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:25.258366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55027fd-5004-429e-9471-9a46c98047fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:25.259172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55027fd-5004-429e-9471-9a46c98047fc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:25.259454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55027fd-5004-429e-9471-9a46c98047fc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:25.259731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55027fd-5004-429e-9471-9a46c98047fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:25.268205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55027fd-5004-429e-9471-9a46c98047fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:25.269677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55027fd-5004-429e-9471-9a46c98047fc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f55027fd-5004-429e-9471-9a46c98047fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:47:25.285250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:25.286577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:25.287744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:25.287744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:25.287744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:47:25.287744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:47:25.287744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:47:25.287744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:47:25.287744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:47:25.287744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:47:25.287744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:25.287744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:25.291402 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 610/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:25.292646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c956a3-9b43-40f2-9242-69353c003d49 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:25.427571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c956a3-9b43-40f2-9242-69353c003d49 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:25.469250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f43ce3-7d87-45ca-88e2-cd018716c655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:25.469943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f43ce3-7d87-45ca-88e2-cd018716c655 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:25.470225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f43ce3-7d87-45ca-88e2-cd018716c655 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:25.470504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f43ce3-7d87-45ca-88e2-cd018716c655 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:25.471361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f43ce3-7d87-45ca-88e2-cd018716c655 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:25.471685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f43ce3-7d87-45ca-88e2-cd018716c655 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:25.473631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f43ce3-7d87-45ca-88e2-cd018716c655 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:25.491995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 603/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jul 11 20:47:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:25.497829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55027fd-5004-429e-9471-9a46c98047fc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:25.613094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55027fd-5004-429e-9471-9a46c98047fc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:25.651829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa36a8-a83a-4743-8555-54a995cff4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:25.652533 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa36a8-a83a-4743-8555-54a995cff4db None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:25.652911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa36a8-a83a-4743-8555-54a995cff4db None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:25.654599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa36a8-a83a-4743-8555-54a995cff4db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:25.686098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:25.727121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa36a8-a83a-4743-8555-54a995cff4db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:25.728887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa36a8-a83a-4743-8555-54a995cff4db None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:25.857957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 611/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:47:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:47:25.865231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f43ce3-7d87-45ca-88e2-cd018716c655 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:25.886117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:25.981364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f43ce3-7d87-45ca-88e2-cd018716c655 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:26.018254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d79d7-9af7-4f91-a024-ccef49156c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:26.019261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d79d7-9af7-4f91-a024-ccef49156c7b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:26.019551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d79d7-9af7-4f91-a024-ccef49156c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:26.020054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d79d7-9af7-4f91-a024-ccef49156c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:26.030469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d79d7-9af7-4f91-a024-ccef49156c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:26.031937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d79d7-9af7-4f91-a024-ccef49156c7b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f12d79d7-9af7-4f91-a024-ccef49156c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:26.050644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:26.051936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:26.053047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:26.053047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:26.053047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:26.053047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:26.053047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 604/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:26.058311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa36a8-a83a-4743-8555-54a995cff4db None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:26.134433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:26.181215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa36a8-a83a-4743-8555-54a995cff4db None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:26.223744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a26692-9e1a-4557-a1f5-c4074c9c4ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:26.224556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a26692-9e1a-4557-a1f5-c4074c9c4ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:26.224825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a26692-9e1a-4557-a1f5-c4074c9c4ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:26.225201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a26692-9e1a-4557-a1f5-c4074c9c4ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:26.226219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a26692-9e1a-4557-a1f5-c4074c9c4ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:26.226663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a26692-9e1a-4557-a1f5-c4074c9c4ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:26.228705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a26692-9e1a-4557-a1f5-c4074c9c4ee9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:26.244972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 612/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:26.253497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d79d7-9af7-4f91-a024-ccef49156c7b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:26.335117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:26.373907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d79d7-9af7-4f91-a024-ccef49156c7b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:26.414584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1924b6e-53dd-43d7-bff6-11b4f25e027a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:26.416064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1924b6e-53dd-43d7-bff6-11b4f25e027a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:26.416629 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1924b6e-53dd-43d7-bff6-11b4f25e027a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:26.417062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1924b6e-53dd-43d7-bff6-11b4f25e027a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:26.425024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1924b6e-53dd-43d7-bff6-11b4f25e027a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:26.428473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1924b6e-53dd-43d7-bff6-11b4f25e027a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e1924b6e-53dd-43d7-bff6-11b4f25e027a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:47:26.443747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:26.445176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:26.451311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:47:26.451311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:47:26.451311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:47:26.451311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:47:26.451311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:47:26.451311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:47:26.451311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:26.451311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:26.451311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 605/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:47:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:26.451885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a26692-9e1a-4557-a1f5-c4074c9c4ee9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:26.583215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a26692-9e1a-4557-a1f5-c4074c9c4ee9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:26.610009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:26.624484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d31fa9-9cdf-4996-beee-1a1740ca082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:26.627604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d31fa9-9cdf-4996-beee-1a1740ca082d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:26.627604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d31fa9-9cdf-4996-beee-1a1740ca082d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:26.627604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d31fa9-9cdf-4996-beee-1a1740ca082d None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:26.627604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d31fa9-9cdf-4996-beee-1a1740ca082d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:26.627604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d31fa9-9cdf-4996-beee-1a1740ca082d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:26.629366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d31fa9-9cdf-4996-beee-1a1740ca082d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:26.649393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 613/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:26.651838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1924b6e-53dd-43d7-bff6-11b4f25e027a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:26.775118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1924b6e-53dd-43d7-bff6-11b4f25e027a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:26.807904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:26.814629 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff1247-f0d6-49d6-8e26-44fb536580b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:26.815306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff1247-f0d6-49d6-8e26-44fb536580b4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:26.815592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff1247-f0d6-49d6-8e26-44fb536580b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:26.815963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff1247-f0d6-49d6-8e26-44fb536580b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:26.817002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff1247-f0d6-49d6-8e26-44fb536580b4 None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=946d11b552754364b55eb495f68ab852) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:26.817322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff1247-f0d6-49d6-8e26-44fb536580b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:26.818792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff1247-f0d6-49d6-8e26-44fb536580b4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:26.825466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 606/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:47:26 2022] GET /identity/v3/role_assignments?user.id=946d11b552754364b55eb495f68ab852&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:28.979745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b23bc-ed04-46ed-ace6-d51a952ea943 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:28.980699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b23bc-ed04-46ed-ace6-d51a952ea943 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:28.980699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b23bc-ed04-46ed-ace6-d51a952ea943 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:28.982692 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 614/1220] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:28.990055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae09bf-54a0-4ae1-a534-bee1eee4a168 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:28.990669 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae09bf-54a0-4ae1-a534-bee1eee4a168 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:28.990755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae09bf-54a0-4ae1-a534-bee1eee4a168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:29.011285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bbae09bf-54a0-4ae1-a534-bee1eee4a168 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:29.012536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bbae09bf-54a0-4ae1-a534-bee1eee4a168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:29.086830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 607/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:47:29.110689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b23bc-ed04-46ed-ace6-d51a952ea943 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:29.222071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b23bc-ed04-46ed-ace6-d51a952ea943 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:29.259122 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec4455-31f2-4e12-9645-1578d7407591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:29.260171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec4455-31f2-4e12-9645-1578d7407591 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:29.260487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec4455-31f2-4e12-9645-1578d7407591 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:29.260762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec4455-31f2-4e12-9645-1578d7407591 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:29.273104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec4455-31f2-4e12-9645-1578d7407591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-adec4455-31f2-4e12-9645-1578d7407591 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:47:29.281986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:29.283278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:29.284148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 615/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:47:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:29.290331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae09bf-54a0-4ae1-a534-bee1eee4a168 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:29.392542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae09bf-54a0-4ae1-a534-bee1eee4a168 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:29.417152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:29.428963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6eb7e1-1382-4383-9066-325309bdb225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:29.429614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6eb7e1-1382-4383-9066-325309bdb225 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:29.429894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6eb7e1-1382-4383-9066-325309bdb225 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:29.430165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6eb7e1-1382-4383-9066-325309bdb225 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:29.431147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6eb7e1-1382-4383-9066-325309bdb225 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:29.431263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6eb7e1-1382-4383-9066-325309bdb225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:29.432668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6eb7e1-1382-4383-9066-325309bdb225 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:29.441209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 608/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:47:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:29.447662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec4455-31f2-4e12-9645-1578d7407591 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:29.496682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:29.562760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec4455-31f2-4e12-9645-1578d7407591 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:29.600688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018645f-4752-4d21-a26a-91f74ae58843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:29.601502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018645f-4752-4d21-a26a-91f74ae58843 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:29.601816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018645f-4752-4d21-a26a-91f74ae58843 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:29.602123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018645f-4752-4d21-a26a-91f74ae58843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:29.614292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018645f-4752-4d21-a26a-91f74ae58843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:29.615965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018645f-4752-4d21-a26a-91f74ae58843 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b018645f-4752-4d21-a26a-91f74ae58843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:29.634479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:29.635607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:29.636753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:29.636753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:29.636753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:29.636753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:29.640263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 616/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:29.641876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6eb7e1-1382-4383-9066-325309bdb225 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:29.750591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6eb7e1-1382-4383-9066-325309bdb225 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:29.788219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a686e34-ab48-495b-9621-3bf782847f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:29.788957 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a686e34-ab48-495b-9621-3bf782847f3d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:29.789297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a686e34-ab48-495b-9621-3bf782847f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:29.789570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a686e34-ab48-495b-9621-3bf782847f3d None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:29.790423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a686e34-ab48-495b-9621-3bf782847f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:29.790760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a686e34-ab48-495b-9621-3bf782847f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:29.792511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a686e34-ab48-495b-9621-3bf782847f3d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:29.806515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 609/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:29.812269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018645f-4752-4d21-a26a-91f74ae58843 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:29.914942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:29.922299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018645f-4752-4d21-a26a-91f74ae58843 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:29.959277 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090106b-1562-44b0-ad90-15d78dc22349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:29.960202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090106b-1562-44b0-ad90-15d78dc22349 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:29.960527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090106b-1562-44b0-ad90-15d78dc22349 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:29.960798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090106b-1562-44b0-ad90-15d78dc22349 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:29.970223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090106b-1562-44b0-ad90-15d78dc22349 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:29.971827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090106b-1562-44b0-ad90-15d78dc22349 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9090106b-1562-44b0-ad90-15d78dc22349 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:47:29.988920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:29.990452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:29.991576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 617/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:29.996439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a686e34-ab48-495b-9621-3bf782847f3d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:30.053010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:30.114309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a686e34-ab48-495b-9621-3bf782847f3d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:30.147731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82f715-344e-4fb2-9424-ba29ee68f70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:30.148463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82f715-344e-4fb2-9424-ba29ee68f70f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:30.148775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82f715-344e-4fb2-9424-ba29ee68f70f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:30.149106 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82f715-344e-4fb2-9424-ba29ee68f70f None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:30.150164 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82f715-344e-4fb2-9424-ba29ee68f70f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:30.150534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82f715-344e-4fb2-9424-ba29ee68f70f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:30.152212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82f715-344e-4fb2-9424-ba29ee68f70f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:30.176042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 610/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jul 11 20:47:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:30.181929 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090106b-1562-44b0-ad90-15d78dc22349 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:30.313239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090106b-1562-44b0-ad90-15d78dc22349 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:30.350940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ca15f-e20a-4b65-bcf8-f3887216bf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:30.351750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ca15f-e20a-4b65-bcf8-f3887216bf06 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:30.352031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ca15f-e20a-4b65-bcf8-f3887216bf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:30.352451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ca15f-e20a-4b65-bcf8-f3887216bf06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:30.363671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ca15f-e20a-4b65-bcf8-f3887216bf06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:30.365325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ca15f-e20a-4b65-bcf8-f3887216bf06 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-008ca15f-e20a-4b65-bcf8-f3887216bf06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:30.385797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:30.386938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:30.388013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:30.388013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:30.388013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:30.388013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:30.388013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 618/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:30.393524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82f715-344e-4fb2-9424-ba29ee68f70f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:30.508396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82f715-344e-4fb2-9424-ba29ee68f70f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:30.511511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:30.558615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece7ac-4e51-4ba2-9af4-a325c89b9f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:30.559288 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece7ac-4e51-4ba2-9af4-a325c89b9f51 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:30.559580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece7ac-4e51-4ba2-9af4-a325c89b9f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:30.559839 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece7ac-4e51-4ba2-9af4-a325c89b9f51 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:30.560827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ece7ac-4e51-4ba2-9af4-a325c89b9f51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:30.561131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ece7ac-4e51-4ba2-9af4-a325c89b9f51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:30.562540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ece7ac-4e51-4ba2-9af4-a325c89b9f51 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:30.579148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 611/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:30.585699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ca15f-e20a-4b65-bcf8-f3887216bf06 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:30.595509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:30.697290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ca15f-e20a-4b65-bcf8-f3887216bf06 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:30.744010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4e022-5698-49a4-a31a-30d507213286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:30.745044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4e022-5698-49a4-a31a-30d507213286 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:30.745422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4e022-5698-49a4-a31a-30d507213286 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:30.745764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4e022-5698-49a4-a31a-30d507213286 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:30.754905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4e022-5698-49a4-a31a-30d507213286 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:30.757826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4e022-5698-49a4-a31a-30d507213286 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-74b4e022-5698-49a4-a31a-30d507213286 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:47:30.777356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:30.778316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:30.779417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:47:30.779417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:47:30.779417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:47:30.779417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:47:30.779417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:47:30.779417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:47:30.779417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:30.779417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:30.779417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 619/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:47:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:30.786499 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece7ac-4e51-4ba2-9af4-a325c89b9f51 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:30.931017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece7ac-4e51-4ba2-9af4-a325c89b9f51 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:30.973385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bfafe-b535-4458-999f-96a0056dde3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:30.973962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bfafe-b535-4458-999f-96a0056dde3b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:30.974244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bfafe-b535-4458-999f-96a0056dde3b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:30.974521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bfafe-b535-4458-999f-96a0056dde3b None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:30.975325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bfafe-b535-4458-999f-96a0056dde3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:30.975638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bfafe-b535-4458-999f-96a0056dde3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:30.977110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bfafe-b535-4458-999f-96a0056dde3b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:30.990355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 612/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:47:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:30.998723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4e022-5698-49a4-a31a-30d507213286 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:31.091167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:31.139046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4e022-5698-49a4-a31a-30d507213286 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:31.184686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9090911-92ef-4fcc-9185-c783747edf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:31.185789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9090911-92ef-4fcc-9185-c783747edf3d None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:31.186150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9090911-92ef-4fcc-9185-c783747edf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:31.186496 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9090911-92ef-4fcc-9185-c783747edf3d None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/946d11b552754364b55eb495f68ab852/roles/91ff1620e1064bf199221383f124aa3c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:31.209531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9090911-92ef-4fcc-9185-c783747edf3d None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=946d11b552754364b55eb495f68ab852, role_id=91ff1620e1064bf199221383f124aa3c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:31.211582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9090911-92ef-4fcc-9185-c783747edf3d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:31.234077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 620/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:47:30 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/946d11b552754364b55eb495f68ab852/roles/91ff1620e1064bf199221383f124aa3c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:47:33.228159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eb2c4-17ea-40f5-975f-ebc4407924c0 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:33.228741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eb2c4-17ea-40f5-975f-ebc4407924c0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:33.228741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eb2c4-17ea-40f5-975f-ebc4407924c0 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:33.230115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 613/1233] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:33.235769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe7c02-634e-453f-aa0a-a220627ed729 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:33.236366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe7c02-634e-453f-aa0a-a220627ed729 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:33.236366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe7c02-634e-453f-aa0a-a220627ed729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:33.252777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9dfe7c02-634e-453f-aa0a-a220627ed729 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:33.254219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9dfe7c02-634e-453f-aa0a-a220627ed729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:33.323702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 621/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:47:33.347062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eb2c4-17ea-40f5-975f-ebc4407924c0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:33.374279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:33.455734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eb2c4-17ea-40f5-975f-ebc4407924c0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:33.493892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536fc8c-a9ed-44fa-891b-4a995eb1b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:33.494630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536fc8c-a9ed-44fa-891b-4a995eb1b311 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:33.494909 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536fc8c-a9ed-44fa-891b-4a995eb1b311 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:33.495179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536fc8c-a9ed-44fa-891b-4a995eb1b311 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:33.505990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536fc8c-a9ed-44fa-891b-4a995eb1b311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f536fc8c-a9ed-44fa-891b-4a995eb1b311 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:47:33.513742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:33.514936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 20:47:33.515960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:33.520028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 614/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jul 11 20:47:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:33.521156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe7c02-634e-453f-aa0a-a220627ed729 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:33.570886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:33.630402 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe7c02-634e-453f-aa0a-a220627ed729 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:33.663777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c99f6-2c44-44ee-ac3c-24750e07f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:33.664488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c99f6-2c44-44ee-ac3c-24750e07f35a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:33.664770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c99f6-2c44-44ee-ac3c-24750e07f35a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:33.665047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c99f6-2c44-44ee-ac3c-24750e07f35a None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:33.665798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c99f6-2c44-44ee-ac3c-24750e07f35a None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:33.666108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c99f6-2c44-44ee-ac3c-24750e07f35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:33.667485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c99f6-2c44-44ee-ac3c-24750e07f35a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:33.676656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 622/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jul 11 20:47:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:33.682742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536fc8c-a9ed-44fa-891b-4a995eb1b311 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:33.767612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:33.785157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536fc8c-a9ed-44fa-891b-4a995eb1b311 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:33.819604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a290341-206d-4f94-a73a-8c5881fcb418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:33.820465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a290341-206d-4f94-a73a-8c5881fcb418 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:33.820745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a290341-206d-4f94-a73a-8c5881fcb418 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:33.821020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a290341-206d-4f94-a73a-8c5881fcb418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:33.835511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a290341-206d-4f94-a73a-8c5881fcb418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:33.837076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a290341-206d-4f94-a73a-8c5881fcb418 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-6a290341-206d-4f94-a73a-8c5881fcb418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:33.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:33.855856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:33.856813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:33.856813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:33.856813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:33.856813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:33.856813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 615/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:33.861938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c99f6-2c44-44ee-ac3c-24750e07f35a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:33.964316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c99f6-2c44-44ee-ac3c-24750e07f35a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:33.973839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:34.001787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3566b2-a85d-4c08-be52-12de4a021d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:34.002632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3566b2-a85d-4c08-be52-12de4a021d31 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:34.003005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3566b2-a85d-4c08-be52-12de4a021d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:34.003465 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3566b2-a85d-4c08-be52-12de4a021d31 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:34.004507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3566b2-a85d-4c08-be52-12de4a021d31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:34.004921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3566b2-a85d-4c08-be52-12de4a021d31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:34.006612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3566b2-a85d-4c08-be52-12de4a021d31 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:34.018357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 623/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:47:34.024226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a290341-206d-4f94-a73a-8c5881fcb418 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:34.131614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a290341-206d-4f94-a73a-8c5881fcb418 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:34.169784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4430a17-437d-4c4a-8419-017dd8ca1bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:34.170551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4430a17-437d-4c4a-8419-017dd8ca1bec None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:34.170823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4430a17-437d-4c4a-8419-017dd8ca1bec None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:34.171099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4430a17-437d-4c4a-8419-017dd8ca1bec None admin] PATH_INFO: `/v3/users/placement` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:34.174178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:34.180266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4430a17-437d-4c4a-8419-017dd8ca1bec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:34.181724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4430a17-437d-4c4a-8419-017dd8ca1bec None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f4430a17-437d-4c4a-8419-017dd8ca1bec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:47:34.197660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:34.198901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:34.200152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 616/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:34.205123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3566b2-a85d-4c08-be52-12de4a021d31 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:34.306829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3566b2-a85d-4c08-be52-12de4a021d31 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:34.340931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b507b43-7452-4c20-aff7-abe37df2a69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:34.341619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507b43-7452-4c20-aff7-abe37df2a69f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:34.341885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507b43-7452-4c20-aff7-abe37df2a69f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:34.342157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507b43-7452-4c20-aff7-abe37df2a69f None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:34.342992 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b507b43-7452-4c20-aff7-abe37df2a69f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:34.343299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b507b43-7452-4c20-aff7-abe37df2a69f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:34.344845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b507b43-7452-4c20-aff7-abe37df2a69f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:34.367889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 624/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jul 11 20:47:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:34.369203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4430a17-437d-4c4a-8419-017dd8ca1bec None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:34.477536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4430a17-437d-4c4a-8419-017dd8ca1bec None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:34.511807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382d522-5a68-471b-90ba-f94b5fada871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:34.512544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382d522-5a68-471b-90ba-f94b5fada871 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:34.512859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382d522-5a68-471b-90ba-f94b5fada871 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:34.513128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382d522-5a68-471b-90ba-f94b5fada871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:34.588569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:34.596122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382d522-5a68-471b-90ba-f94b5fada871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:34.597700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382d522-5a68-471b-90ba-f94b5fada871 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:34.705982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 617/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:47:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:47:34.707597 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b507b43-7452-4c20-aff7-abe37df2a69f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:34.738505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:34.813323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b507b43-7452-4c20-aff7-abe37df2a69f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:34.850137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f41822-1c49-4dcc-8d4d-add85df56c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:34.850868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f41822-1c49-4dcc-8d4d-add85df56c40 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:34.851142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f41822-1c49-4dcc-8d4d-add85df56c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:34.851415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f41822-1c49-4dcc-8d4d-add85df56c40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:34.861527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f41822-1c49-4dcc-8d4d-add85df56c40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:34.863164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f41822-1c49-4dcc-8d4d-add85df56c40 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-78f41822-1c49-4dcc-8d4d-add85df56c40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:34.880585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:34.881527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:34.882432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:34.882432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:34.882432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:34.882432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:34.882432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 625/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:34.888947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382d522-5a68-471b-90ba-f94b5fada871 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:34.990700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:34.996673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382d522-5a68-471b-90ba-f94b5fada871 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:35.033980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88e41d-65b9-41bc-b15e-39ad1a16c4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:35.035276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88e41d-65b9-41bc-b15e-39ad1a16c4ad None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:35.035570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88e41d-65b9-41bc-b15e-39ad1a16c4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:35.035841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88e41d-65b9-41bc-b15e-39ad1a16c4ad None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:35.036724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88e41d-65b9-41bc-b15e-39ad1a16c4ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:35.037275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88e41d-65b9-41bc-b15e-39ad1a16c4ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:35.039007 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88e41d-65b9-41bc-b15e-39ad1a16c4ad None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:35.058943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 618/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:47:35.060366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f41822-1c49-4dcc-8d4d-add85df56c40 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:35.146184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:35.168689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f41822-1c49-4dcc-8d4d-add85df56c40 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:35.205706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca332b5-156b-4319-9bfb-6088315dfbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:35.206829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca332b5-156b-4319-9bfb-6088315dfbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:35.207156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca332b5-156b-4319-9bfb-6088315dfbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:35.207536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca332b5-156b-4319-9bfb-6088315dfbd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:35.217903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca332b5-156b-4319-9bfb-6088315dfbd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:35.219370 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca332b5-156b-4319-9bfb-6088315dfbd9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5ca332b5-156b-4319-9bfb-6088315dfbd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:47:35.233504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:35.234555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:35.235507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:47:35.235507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:47:35.235507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:47:35.235507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:47:35.235507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:47:35.235507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:47:35.235507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:35.235507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:35.235507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 626/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:47:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:35.241400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88e41d-65b9-41bc-b15e-39ad1a16c4ad None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:35.363363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88e41d-65b9-41bc-b15e-39ad1a16c4ad None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:35.400842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd50fc4-3e1d-4bf7-b68f-6a91706828d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:35.401509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd50fc4-3e1d-4bf7-b68f-6a91706828d5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:35.401733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd50fc4-3e1d-4bf7-b68f-6a91706828d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:35.401994 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd50fc4-3e1d-4bf7-b68f-6a91706828d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:35.402860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd50fc4-3e1d-4bf7-b68f-6a91706828d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:35.403112 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd50fc4-3e1d-4bf7-b68f-6a91706828d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:35.404606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd50fc4-3e1d-4bf7-b68f-6a91706828d5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:35.418816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 619/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:47:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:35.425472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca332b5-156b-4319-9bfb-6088315dfbd9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:35.534084 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca332b5-156b-4319-9bfb-6088315dfbd9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:35.578097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e489e77-2c69-4327-974f-708ea79531b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:35.578983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e489e77-2c69-4327-974f-708ea79531b9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:35.579378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e489e77-2c69-4327-974f-708ea79531b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:35.579716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e489e77-2c69-4327-974f-708ea79531b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:35.580984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e489e77-2c69-4327-974f-708ea79531b9 None admin] RBAC: Adding query filter params (role.id=91ff1620e1064bf199221383f124aa3c, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=946d11b552754364b55eb495f68ab852) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:35.581573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e489e77-2c69-4327-974f-708ea79531b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:35.583500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e489e77-2c69-4327-974f-708ea79531b9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:35.592705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 627/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:47:35 2022] GET /identity/v3/role_assignments?user.id=946d11b552754364b55eb495f68ab852&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=91ff1620e1064bf199221383f124aa3c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:37.710299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f3610-ef3a-4860-9851-e27f6e293582 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:37.710886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f3610-ef3a-4860-9851-e27f6e293582 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:37.710886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f3610-ef3a-4860-9851-e27f6e293582 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:37.712263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 620/1247] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:37.718062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baf2cc-8659-4288-89fe-7e182fb4eaeb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:37.718588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baf2cc-8659-4288-89fe-7e182fb4eaeb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:37.718588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baf2cc-8659-4288-89fe-7e182fb4eaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:37.725276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:37.741008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-82baf2cc-8659-4288-89fe-7e182fb4eaeb None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:37.742268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-82baf2cc-8659-4288-89fe-7e182fb4eaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:37.817204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 628/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:47:37.840292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f3610-ef3a-4860-9851-e27f6e293582 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:37.861035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:37.954711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f3610-ef3a-4860-9851-e27f6e293582 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:37.992023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902f22a-8ec8-46b6-815c-421c1edb10ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:37.993118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902f22a-8ec8-46b6-815c-421c1edb10ac None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:37.993561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902f22a-8ec8-46b6-815c-421c1edb10ac None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:37.993861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902f22a-8ec8-46b6-815c-421c1edb10ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:38.004997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902f22a-8ec8-46b6-815c-421c1edb10ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-5902f22a-8ec8-46b6-815c-421c1edb10ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:47:38.012626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:38.013738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:38.014632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 621/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:47:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:47:38.020164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baf2cc-8659-4288-89fe-7e182fb4eaeb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:38.127385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baf2cc-8659-4288-89fe-7e182fb4eaeb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:38.157415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:38.162880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f924097-a183-4c47-b91d-d349f5fcc254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:38.163562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f924097-a183-4c47-b91d-d349f5fcc254 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:38.163856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f924097-a183-4c47-b91d-d349f5fcc254 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:38.164159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f924097-a183-4c47-b91d-d349f5fcc254 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:38.164919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f924097-a183-4c47-b91d-d349f5fcc254 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:38.165253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f924097-a183-4c47-b91d-d349f5fcc254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:38.166646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f924097-a183-4c47-b91d-d349f5fcc254 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:38.175324 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 629/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:47:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:38.181144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902f22a-8ec8-46b6-815c-421c1edb10ac None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:38.265687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:38.294019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902f22a-8ec8-46b6-815c-421c1edb10ac None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:38.329978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf126b-43ac-4720-98df-00e56f4b733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:38.330821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf126b-43ac-4720-98df-00e56f4b733a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:38.331147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf126b-43ac-4720-98df-00e56f4b733a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:38.331460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf126b-43ac-4720-98df-00e56f4b733a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:38.343908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf126b-43ac-4720-98df-00e56f4b733a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:38.345400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf126b-43ac-4720-98df-00e56f4b733a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-4cbf126b-43ac-4720-98df-00e56f4b733a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:38.363201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:38.364406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:38.365330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:38.365330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:38.365330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:38.365330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:38.365330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 622/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:38.370429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f924097-a183-4c47-b91d-d349f5fcc254 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:38.482951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f924097-a183-4c47-b91d-d349f5fcc254 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:38.521023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dfd4fa-c761-433f-8c3c-c300371becad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:38.521619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dfd4fa-c761-433f-8c3c-c300371becad None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:38.521905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dfd4fa-c761-433f-8c3c-c300371becad None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:38.522184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dfd4fa-c761-433f-8c3c-c300371becad None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:38.522910 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dfd4fa-c761-433f-8c3c-c300371becad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:38.523215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dfd4fa-c761-433f-8c3c-c300371becad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:38.524645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dfd4fa-c761-433f-8c3c-c300371becad None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:38.538671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 630/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:38.544294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf126b-43ac-4720-98df-00e56f4b733a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:38.655730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf126b-43ac-4720-98df-00e56f4b733a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:38.684515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:38.693528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeeccf-1460-4241-addb-47de3d41a1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:38.694702 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeeccf-1460-4241-addb-47de3d41a1ee None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:38.694977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeeccf-1460-4241-addb-47de3d41a1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:38.695242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeeccf-1460-4241-addb-47de3d41a1ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:38.703152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdeeccf-1460-4241-addb-47de3d41a1ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:38.704659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdeeccf-1460-4241-addb-47de3d41a1ee None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-8cdeeccf-1460-4241-addb-47de3d41a1ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:47:38.720035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:38.721630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:38.722888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 623/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:38.727275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dfd4fa-c761-433f-8c3c-c300371becad None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:38.789991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:38.840236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dfd4fa-c761-433f-8c3c-c300371becad None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:38.876545 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69266cb3-6c8b-4e56-ad7d-4cd61b8a0b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:38.877358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69266cb3-6c8b-4e56-ad7d-4cd61b8a0b28 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:38.877644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69266cb3-6c8b-4e56-ad7d-4cd61b8a0b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:38.877919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69266cb3-6c8b-4e56-ad7d-4cd61b8a0b28 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:38.878828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69266cb3-6c8b-4e56-ad7d-4cd61b8a0b28 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:38.879133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69266cb3-6c8b-4e56-ad7d-4cd61b8a0b28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:38.880605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69266cb3-6c8b-4e56-ad7d-4cd61b8a0b28 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:38.900562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 631/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jul 11 20:47:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:47:38.906634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeeccf-1460-4241-addb-47de3d41a1ee None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:39.024130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeeccf-1460-4241-addb-47de3d41a1ee None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:39.067381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d67c6-15c3-4ea5-9416-5484dff893a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:39.068327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d67c6-15c3-4ea5-9416-5484dff893a5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:39.068611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d67c6-15c3-4ea5-9416-5484dff893a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:39.069246 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d67c6-15c3-4ea5-9416-5484dff893a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:39.128787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:39.142495 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d67c6-15c3-4ea5-9416-5484dff893a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:39.144156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d67c6-15c3-4ea5-9416-5484dff893a5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:39.265575 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 624/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:47:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:47:39.271503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69266cb3-6c8b-4e56-ad7d-4cd61b8a0b28 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:39.394731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69266cb3-6c8b-4e56-ad7d-4cd61b8a0b28 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:39.410418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:39.433967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b48dc3-2007-41d8-8d25-fb9aac06ba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:39.434901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b48dc3-2007-41d8-8d25-fb9aac06ba40 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:39.435197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b48dc3-2007-41d8-8d25-fb9aac06ba40 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:39.435485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b48dc3-2007-41d8-8d25-fb9aac06ba40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:39.448648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b48dc3-2007-41d8-8d25-fb9aac06ba40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:39.450121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b48dc3-2007-41d8-8d25-fb9aac06ba40 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-56b48dc3-2007-41d8-8d25-fb9aac06ba40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:39.470762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:39.472385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:39.473696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:39.473696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:39.473696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:39.473696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:39.474064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 632/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:39.480429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d67c6-15c3-4ea5-9416-5484dff893a5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:39.580096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:39.594478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d67c6-15c3-4ea5-9416-5484dff893a5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:39.633186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b7945-b7da-4f7b-bf0f-7a8ed7f847fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:39.633972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b7945-b7da-4f7b-bf0f-7a8ed7f847fd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:39.634247 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b7945-b7da-4f7b-bf0f-7a8ed7f847fd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:39.634588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b7945-b7da-4f7b-bf0f-7a8ed7f847fd None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:39.635331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b7945-b7da-4f7b-bf0f-7a8ed7f847fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:39.635641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b7945-b7da-4f7b-bf0f-7a8ed7f847fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:39.637290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b7945-b7da-4f7b-bf0f-7a8ed7f847fd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:39.649812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 625/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:39.655623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b48dc3-2007-41d8-8d25-fb9aac06ba40 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:39.761558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b48dc3-2007-41d8-8d25-fb9aac06ba40 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:39.797214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a051e7a-1dac-4267-bcb5-9c352e6e79fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:39.798037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051e7a-1dac-4267-bcb5-9c352e6e79fa None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:39.798313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051e7a-1dac-4267-bcb5-9c352e6e79fa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:39.798593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051e7a-1dac-4267-bcb5-9c352e6e79fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:39.805853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a051e7a-1dac-4267-bcb5-9c352e6e79fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:39.807321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a051e7a-1dac-4267-bcb5-9c352e6e79fa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-4a051e7a-1dac-4267-bcb5-9c352e6e79fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:47:39.823905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:39.825070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:39.826702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:47:39.826702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:47:39.826702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:47:39.826702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:47:39.826702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:47:39.826702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:47:39.826702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:39.826702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:39.826702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 633/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:39.832326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b7945-b7da-4f7b-bf0f-7a8ed7f847fd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:39.941498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b7945-b7da-4f7b-bf0f-7a8ed7f847fd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:39.982229 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50242b5-2ca3-4f1c-9847-3e4ef6001f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:39.982909 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50242b5-2ca3-4f1c-9847-3e4ef6001f81 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:39.983187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50242b5-2ca3-4f1c-9847-3e4ef6001f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:39.983459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50242b5-2ca3-4f1c-9847-3e4ef6001f81 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:39.984336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50242b5-2ca3-4f1c-9847-3e4ef6001f81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:39.984653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50242b5-2ca3-4f1c-9847-3e4ef6001f81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:39.987429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50242b5-2ca3-4f1c-9847-3e4ef6001f81 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:40.000507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 626/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:40.007803 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a051e7a-1dac-4267-bcb5-9c352e6e79fa None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:40.037150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:40.123706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a051e7a-1dac-4267-bcb5-9c352e6e79fa None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:40.164760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88701d2-b714-401d-8e31-f9a8e526b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:40.165430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88701d2-b714-401d-8e31-f9a8e526b476 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:40.165714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88701d2-b714-401d-8e31-f9a8e526b476 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:40.165988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88701d2-b714-401d-8e31-f9a8e526b476 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:40.166827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88701d2-b714-401d-8e31-f9a8e526b476 None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=946d11b552754364b55eb495f68ab852) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:40.167138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88701d2-b714-401d-8e31-f9a8e526b476 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:40.168627 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88701d2-b714-401d-8e31-f9a8e526b476 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:40.174300 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 634/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:47:40 2022] GET /identity/v3/role_assignments?user.id=946d11b552754364b55eb495f68ab852&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=60bd1c44ce154338a7face8949d07281 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:42.168519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae336d33-eae8-46c9-91c9-63809b8aa928 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:42.169094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae336d33-eae8-46c9-91c9-63809b8aa928 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:42.169094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae336d33-eae8-46c9-91c9-63809b8aa928 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:42.170484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 627/1261] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:42.176922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c897d5f-f691-45a8-821f-fe16f0b9420c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:42.177600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c897d5f-f691-45a8-821f-fe16f0b9420c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:42.177743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c897d5f-f691-45a8-821f-fe16f0b9420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:42.198330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0c897d5f-f691-45a8-821f-fe16f0b9420c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:42.199507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c897d5f-f691-45a8-821f-fe16f0b9420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:42.289987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 635/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:47:42.323532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae336d33-eae8-46c9-91c9-63809b8aa928 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:42.338929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:42.454522 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae336d33-eae8-46c9-91c9-63809b8aa928 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:42.490145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39ad02-4f51-4f6b-a975-5259205e6993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:42.490945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39ad02-4f51-4f6b-a975-5259205e6993 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:42.491220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39ad02-4f51-4f6b-a975-5259205e6993 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:42.491494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39ad02-4f51-4f6b-a975-5259205e6993 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:42.502629 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39ad02-4f51-4f6b-a975-5259205e6993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9b39ad02-4f51-4f6b-a975-5259205e6993 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:47:42.510188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:42.511271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:42.512505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 628/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:47:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:42.521217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c897d5f-f691-45a8-821f-fe16f0b9420c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:42.544931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:42.630186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c897d5f-f691-45a8-821f-fe16f0b9420c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:42.667368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12a6ab-6d1c-4859-a921-89e25607b01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:42.668049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12a6ab-6d1c-4859-a921-89e25607b01c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:42.668363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12a6ab-6d1c-4859-a921-89e25607b01c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:42.668642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12a6ab-6d1c-4859-a921-89e25607b01c None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:42.669397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12a6ab-6d1c-4859-a921-89e25607b01c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:42.669706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12a6ab-6d1c-4859-a921-89e25607b01c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:42.671140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12a6ab-6d1c-4859-a921-89e25607b01c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:42.680459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 636/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:47:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:42.687082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39ad02-4f51-4f6b-a975-5259205e6993 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:42.762375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:42.804480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39ad02-4f51-4f6b-a975-5259205e6993 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:42.843460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884efbb5-36d0-4faa-b3e1-f47202cc933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:42.844307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884efbb5-36d0-4faa-b3e1-f47202cc933a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:42.844583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884efbb5-36d0-4faa-b3e1-f47202cc933a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:42.844854 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884efbb5-36d0-4faa-b3e1-f47202cc933a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:42.855438 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884efbb5-36d0-4faa-b3e1-f47202cc933a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:42.856913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884efbb5-36d0-4faa-b3e1-f47202cc933a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-884efbb5-36d0-4faa-b3e1-f47202cc933a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:42.873546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:42.874583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:42.875479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:42.875479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:42.875479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:42.875479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:42.875479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 629/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:42.880991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12a6ab-6d1c-4859-a921-89e25607b01c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:42.978179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:43.001698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12a6ab-6d1c-4859-a921-89e25607b01c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:43.042679 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240101a-b045-4e15-b54d-9334dfc97caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:43.043560 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240101a-b045-4e15-b54d-9334dfc97caf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:43.043981 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240101a-b045-4e15-b54d-9334dfc97caf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:43.044424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240101a-b045-4e15-b54d-9334dfc97caf None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:43.045554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240101a-b045-4e15-b54d-9334dfc97caf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:43.046016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240101a-b045-4e15-b54d-9334dfc97caf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:43.048210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240101a-b045-4e15-b54d-9334dfc97caf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:43.062099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 637/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:43.070382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884efbb5-36d0-4faa-b3e1-f47202cc933a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:43.180440 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884efbb5-36d0-4faa-b3e1-f47202cc933a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:43.200377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:43.216050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a458c-a8f2-4aec-bd9b-b62cf9a07969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:43.216882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a458c-a8f2-4aec-bd9b-b62cf9a07969 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:43.217156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a458c-a8f2-4aec-bd9b-b62cf9a07969 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:43.217424 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a458c-a8f2-4aec-bd9b-b62cf9a07969 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:43.228421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a458c-a8f2-4aec-bd9b-b62cf9a07969 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:43.229860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a458c-a8f2-4aec-bd9b-b62cf9a07969 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-0a1a458c-a8f2-4aec-bd9b-b62cf9a07969 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:47:43.244617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:43.245698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:43.246904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:43.246904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:43.246904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:47:43.246904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:47:43.246904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:47:43.246904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:47:43.246904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:47:43.246904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:47:43.246904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:43.246904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:43.248494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 630/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:43.252254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240101a-b045-4e15-b54d-9334dfc97caf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:43.363476 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240101a-b045-4e15-b54d-9334dfc97caf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:43.406798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415babae-09dd-4c53-816a-9ed9605205ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:43.407770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415babae-09dd-4c53-816a-9ed9605205ba None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:43.408215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415babae-09dd-4c53-816a-9ed9605205ba None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:43.408627 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415babae-09dd-4c53-816a-9ed9605205ba None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:43.409997 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415babae-09dd-4c53-816a-9ed9605205ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:43.410390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415babae-09dd-4c53-816a-9ed9605205ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:43.412674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415babae-09dd-4c53-816a-9ed9605205ba None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:43.424058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:43.435192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 638/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jul 11 20:47:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:43.441335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a458c-a8f2-4aec-bd9b-b62cf9a07969 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:43.554217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a458c-a8f2-4aec-bd9b-b62cf9a07969 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:43.591305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a902b5-deae-42cd-826f-f548ec3f3828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:43.592081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a902b5-deae-42cd-826f-f548ec3f3828 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:43.592390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a902b5-deae-42cd-826f-f548ec3f3828 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:43.592664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a902b5-deae-42cd-826f-f548ec3f3828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:43.603812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a902b5-deae-42cd-826f-f548ec3f3828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:43.605681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a902b5-deae-42cd-826f-f548ec3f3828 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a5a902b5-deae-42cd-826f-f548ec3f3828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:43.623073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:43.624056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:43.625122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:43.625122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:43.625122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:43.625122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:43.625122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 631/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 20:47:43.630082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415babae-09dd-4c53-816a-9ed9605205ba None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:43.742866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415babae-09dd-4c53-816a-9ed9605205ba None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:43.778054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1e304-09bd-4471-bf10-d2560c637744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:43.778703 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1e304-09bd-4471-bf10-d2560c637744 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:43.778985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1e304-09bd-4471-bf10-d2560c637744 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:43.779254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1e304-09bd-4471-bf10-d2560c637744 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:43.779976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1e304-09bd-4471-bf10-d2560c637744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:43.780310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1e304-09bd-4471-bf10-d2560c637744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:43.781683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1e304-09bd-4471-bf10-d2560c637744 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:43.792807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 639/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:43.801538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a902b5-deae-42cd-826f-f548ec3f3828 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:43.837119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:43.906367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a902b5-deae-42cd-826f-f548ec3f3828 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:43.942820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec1db-da2d-407e-b3f5-d124e2955351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:43.943962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec1db-da2d-407e-b3f5-d124e2955351 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:43.944252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec1db-da2d-407e-b3f5-d124e2955351 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:43.944523 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec1db-da2d-407e-b3f5-d124e2955351 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:43.951502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fec1db-da2d-407e-b3f5-d124e2955351 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:43.952981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fec1db-da2d-407e-b3f5-d124e2955351 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f3fec1db-da2d-407e-b3f5-d124e2955351 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:47:43.967159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:43.968187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:43.973283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:47:43.973283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:47:43.973283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:47:43.973283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:47:43.973283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:47:43.973283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:47:43.973283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:43.973283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:43.973283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 632/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:47:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:43.976093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1e304-09bd-4471-bf10-d2560c637744 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:44.048040 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:44.090195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1e304-09bd-4471-bf10-d2560c637744 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:44.125362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea17ba4-deef-41a5-86a1-5248f5b2800e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:44.126011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea17ba4-deef-41a5-86a1-5248f5b2800e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:44.126261 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea17ba4-deef-41a5-86a1-5248f5b2800e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:44.126550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea17ba4-deef-41a5-86a1-5248f5b2800e None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:44.127340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea17ba4-deef-41a5-86a1-5248f5b2800e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:44.127652 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea17ba4-deef-41a5-86a1-5248f5b2800e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:44.129228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea17ba4-deef-41a5-86a1-5248f5b2800e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:44.141195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 640/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:47:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:44.147426 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec1db-da2d-407e-b3f5-d124e2955351 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:44.249607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:44.259425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec1db-da2d-407e-b3f5-d124e2955351 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:44.293840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc0f28-be4d-4613-8d06-c73127190ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:44.294721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc0f28-be4d-4613-8d06-c73127190ef1 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:44.294993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc0f28-be4d-4613-8d06-c73127190ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:44.295272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc0f28-be4d-4613-8d06-c73127190ef1 None admin] PATH_INFO: `/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/946d11b552754364b55eb495f68ab852/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:44.297335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc0f28-be4d-4613-8d06-c73127190ef1 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc0dee82ab60459fa58fc3fcffc09043, user_id=946d11b552754364b55eb495f68ab852, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:44.298768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc0f28-be4d-4613-8d06-c73127190ef1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:44.316031 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 633/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jul 11 20:47:44 2022] PUT /identity/v3/projects/cc0dee82ab60459fa58fc3fcffc09043/users/946d11b552754364b55eb495f68ab852/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:47:46.316399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79aee7-d31d-4cd5-99b2-19657318e90d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:46.317248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79aee7-d31d-4cd5-99b2-19657318e90d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:46.317248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79aee7-d31d-4cd5-99b2-19657318e90d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:46.319934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 641/1274] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:46.327709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8adb5-12da-4e37-a58f-2c48a28a438f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:46.328175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8adb5-12da-4e37-a58f-2c48a28a438f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:46.328271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8adb5-12da-4e37-a58f-2c48a28a438f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:46.347501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-23e8adb5-12da-4e37-a58f-2c48a28a438f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:46.348727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-23e8adb5-12da-4e37-a58f-2c48a28a438f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:46.431732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 634/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 20:47:46.455339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79aee7-d31d-4cd5-99b2-19657318e90d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:46.572269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79aee7-d31d-4cd5-99b2-19657318e90d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:46.590004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:46.607427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cbb9b-a2af-44e1-b764-76f22b47dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:46.608275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cbb9b-a2af-44e1-b764-76f22b47dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:46.608562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cbb9b-a2af-44e1-b764-76f22b47dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:46.608839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cbb9b-a2af-44e1-b764-76f22b47dbc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:46.619379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cbb9b-a2af-44e1-b764-76f22b47dbc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-aa8cbb9b-a2af-44e1-b764-76f22b47dbc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 20:47:46.627066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:46.628141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:46.629102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 642/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jul 11 20:47:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:46.634700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8adb5-12da-4e37-a58f-2c48a28a438f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:46.722587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:46.742487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8adb5-12da-4e37-a58f-2c48a28a438f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:46.785473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e466e3-dbe7-4205-b8b5-8c1e3db4d1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:46.786142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e466e3-dbe7-4205-b8b5-8c1e3db4d1ec None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:46.786401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e466e3-dbe7-4205-b8b5-8c1e3db4d1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:46.786672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e466e3-dbe7-4205-b8b5-8c1e3db4d1ec None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:46.787435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e466e3-dbe7-4205-b8b5-8c1e3db4d1ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:46.787741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e466e3-dbe7-4205-b8b5-8c1e3db4d1ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:46.789167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e466e3-dbe7-4205-b8b5-8c1e3db4d1ec None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:46.798138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 635/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jul 11 20:47:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:46.807827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cbb9b-a2af-44e1-b764-76f22b47dbc1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:46.917007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cbb9b-a2af-44e1-b764-76f22b47dbc1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:46.952000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e221bd-5e7e-46cc-b2e1-19e3fa069d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:46.952827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e221bd-5e7e-46cc-b2e1-19e3fa069d1e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:46.953106 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e221bd-5e7e-46cc-b2e1-19e3fa069d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:46.953382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e221bd-5e7e-46cc-b2e1-19e3fa069d1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:46.965165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e221bd-5e7e-46cc-b2e1-19e3fa069d1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:46.966618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e221bd-5e7e-46cc-b2e1-19e3fa069d1e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b9e221bd-5e7e-46cc-b2e1-19e3fa069d1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:46.983791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:46.984793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:46.985930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:46.985930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:46.985930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:46.985930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:46.985930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 643/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:46.994119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e466e3-dbe7-4205-b8b5-8c1e3db4d1ec None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:47.101657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e466e3-dbe7-4205-b8b5-8c1e3db4d1ec None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:47.137135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae460d70-7d41-471b-ab68-389658e8d4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:47.137795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae460d70-7d41-471b-ab68-389658e8d4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:47.138073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae460d70-7d41-471b-ab68-389658e8d4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:47.138345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae460d70-7d41-471b-ab68-389658e8d4e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:47.139072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae460d70-7d41-471b-ab68-389658e8d4e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:47.139377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae460d70-7d41-471b-ab68-389658e8d4e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:47.140784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae460d70-7d41-471b-ab68-389658e8d4e0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:47.152662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 636/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:47.160646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e221bd-5e7e-46cc-b2e1-19e3fa069d1e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:47.196326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:47.278659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e221bd-5e7e-46cc-b2e1-19e3fa069d1e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:47.313182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea82374-3601-4fc3-8446-b56c16e5e199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:47.313990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea82374-3601-4fc3-8446-b56c16e5e199 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:47.314258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea82374-3601-4fc3-8446-b56c16e5e199 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:47.314529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea82374-3601-4fc3-8446-b56c16e5e199 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:47.322625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea82374-3601-4fc3-8446-b56c16e5e199 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:47.324063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea82374-3601-4fc3-8446-b56c16e5e199 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-fea82374-3601-4fc3-8446-b56c16e5e199 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 20:47:47.338461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:47.339516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:47.340538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 644/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:47.347320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae460d70-7d41-471b-ab68-389658e8d4e0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:47.351374 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:47.451875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae460d70-7d41-471b-ab68-389658e8d4e0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:47.489717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973064b8-2930-459e-acc1-c51dc8050125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:47.490378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973064b8-2930-459e-acc1-c51dc8050125 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:47.490653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973064b8-2930-459e-acc1-c51dc8050125 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:47.490917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973064b8-2930-459e-acc1-c51dc8050125 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:47.491765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973064b8-2930-459e-acc1-c51dc8050125 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:47.492067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973064b8-2930-459e-acc1-c51dc8050125 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:47.493525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973064b8-2930-459e-acc1-c51dc8050125 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:47.512426 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 637/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jul 11 20:47:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:47:47.518975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea82374-3601-4fc3-8446-b56c16e5e199 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:47.618840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:47.634729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea82374-3601-4fc3-8446-b56c16e5e199 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:47.674393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb3fdd-bf8c-4d52-9bff-1762df1c999a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:47.675154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb3fdd-bf8c-4d52-9bff-1762df1c999a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:47.675531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb3fdd-bf8c-4d52-9bff-1762df1c999a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:47.675801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb3fdd-bf8c-4d52-9bff-1762df1c999a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:47.748226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb3fdd-bf8c-4d52-9bff-1762df1c999a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:47.749642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb3fdd-bf8c-4d52-9bff-1762df1c999a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:47.854964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:47.867069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 645/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jul 11 20:47:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 20:47:47.876259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973064b8-2930-459e-acc1-c51dc8050125 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:47.947208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:47.985921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973064b8-2930-459e-acc1-c51dc8050125 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:48.022363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b305b9c-1481-4425-ad40-98b211d5a00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:48.023088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b305b9c-1481-4425-ad40-98b211d5a00e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:48.023363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b305b9c-1481-4425-ad40-98b211d5a00e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:48.023637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b305b9c-1481-4425-ad40-98b211d5a00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:48.033850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b305b9c-1481-4425-ad40-98b211d5a00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:48.035341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b305b9c-1481-4425-ad40-98b211d5a00e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-2b305b9c-1481-4425-ad40-98b211d5a00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 20:47:48.052925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:48.053999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:48.054996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 20:47:48.054996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 20:47:48.054996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 20:47:48.054996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:48.054996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 638/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:47:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:48.060089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb3fdd-bf8c-4d52-9bff-1762df1c999a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:48.182421 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb3fdd-bf8c-4d52-9bff-1762df1c999a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:48.221491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c85f3-e663-425b-9797-8bb89cc04f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:48.222299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c85f3-e663-425b-9797-8bb89cc04f23 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:48.222583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c85f3-e663-425b-9797-8bb89cc04f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:48.222861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c85f3-e663-425b-9797-8bb89cc04f23 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:48.223588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c85f3-e663-425b-9797-8bb89cc04f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:48.223895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c85f3-e663-425b-9797-8bb89cc04f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:48.225309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c85f3-e663-425b-9797-8bb89cc04f23 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:48.236592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 646/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jul 11 20:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:48.242745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b305b9c-1481-4425-ad40-98b211d5a00e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:48.349869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b305b9c-1481-4425-ad40-98b211d5a00e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:48.365953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:48.390339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603212a-5568-4c62-b5d1-ee2a216102c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:48.391131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603212a-5568-4c62-b5d1-ee2a216102c9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:48.391402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603212a-5568-4c62-b5d1-ee2a216102c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:48.391681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603212a-5568-4c62-b5d1-ee2a216102c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:48.398700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603212a-5568-4c62-b5d1-ee2a216102c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:48.400197 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603212a-5568-4c62-b5d1-ee2a216102c9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a603212a-5568-4c62-b5d1-ee2a216102c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 20:47:48.413921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:48.414988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:48.415993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 20:47:48.415993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 20:47:48.415993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 20:47:48.415993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 20:47:48.415993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 20:47:48.415993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 20:47:48.415993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 20:47:48.415993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Jul 11 20:47:48.415993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 639/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jul 11 20:47:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:48.421237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c85f3-e663-425b-9797-8bb89cc04f23 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:48.472400 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:48.523485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c85f3-e663-425b-9797-8bb89cc04f23 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:48.559261 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbe654-4055-4832-b809-daecedec9268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:48.559926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbe654-4055-4832-b809-daecedec9268 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:48.560261 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbe654-4055-4832-b809-daecedec9268 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:48.560540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbe654-4055-4832-b809-daecedec9268 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:48.561330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbe654-4055-4832-b809-daecedec9268 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:48.561637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbe654-4055-4832-b809-daecedec9268 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:48.563111 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbe654-4055-4832-b809-daecedec9268 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:48.575710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 647/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jul 11 20:47:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:48.581942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603212a-5568-4c62-b5d1-ee2a216102c9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:48.685199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603212a-5568-4c62-b5d1-ee2a216102c9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:48.720751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909869a-7be4-4f30-b816-5716d2bc3723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:48.721397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909869a-7be4-4f30-b816-5716d2bc3723 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:48.721846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909869a-7be4-4f30-b816-5716d2bc3723 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:48.722157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909869a-7be4-4f30-b816-5716d2bc3723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:48.723046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909869a-7be4-4f30-b816-5716d2bc3723 None admin] RBAC: Adding query filter params (role.id=60bd1c44ce154338a7face8949d07281, scope.project.id=cc0dee82ab60459fa58fc3fcffc09043, user.id=946d11b552754364b55eb495f68ab852) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:48.723354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909869a-7be4-4f30-b816-5716d2bc3723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:48.724815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909869a-7be4-4f30-b816-5716d2bc3723 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:48.730698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 640/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jul 11 20:47:48 2022] GET /identity/v3/role_assignments?user.id=946d11b552754364b55eb495f68ab852&scope.project.id=cc0dee82ab60459fa58fc3fcffc09043&role.id=60bd1c44ce154338a7face8949d07281 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:50.731181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64b454-fcd8-4544-9c8a-f825d8193e43 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:50.731775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64b454-fcd8-4544-9c8a-f825d8193e43 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:50.731775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64b454-fcd8-4544-9c8a-f825d8193e43 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:50.733115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 648/1288] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:50.739519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356aaac1-5340-40b0-99a9-0f18c0102ae1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:50.740306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356aaac1-5340-40b0-99a9-0f18c0102ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:50.740409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356aaac1-5340-40b0-99a9-0f18c0102ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:50.763250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-356aaac1-5340-40b0-99a9-0f18c0102ae1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:50.764501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-356aaac1-5340-40b0-99a9-0f18c0102ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:50.795624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:50.838932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 641/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:47:50.861675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64b454-fcd8-4544-9c8a-f825d8193e43 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:50.972995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64b454-fcd8-4544-9c8a-f825d8193e43 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:50.983785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:51.008863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77668f59-6cae-4c3e-8060-061da340c157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:51.009650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77668f59-6cae-4c3e-8060-061da340c157 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:51.009921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77668f59-6cae-4c3e-8060-061da340c157 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:51.010189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77668f59-6cae-4c3e-8060-061da340c157 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:51.010918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77668f59-6cae-4c3e-8060-061da340c157 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:51.012316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77668f59-6cae-4c3e-8060-061da340c157 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-77668f59-6cae-4c3e-8060-061da340c157 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:51.023975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:51.025189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:51.026054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 649/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jul 11 20:47:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:51.032068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356aaac1-5340-40b0-99a9-0f18c0102ae1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:51.133599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356aaac1-5340-40b0-99a9-0f18c0102ae1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:51.172852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2079971-9a78-42ef-9945-63e715dd9da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:51.173518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2079971-9a78-42ef-9945-63e715dd9da3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:51.173795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2079971-9a78-42ef-9945-63e715dd9da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:51.174066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2079971-9a78-42ef-9945-63e715dd9da3 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:51.174886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2079971-9a78-42ef-9945-63e715dd9da3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:51.175190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2079971-9a78-42ef-9945-63e715dd9da3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:51.176602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2079971-9a78-42ef-9945-63e715dd9da3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:51.181673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 642/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Jul 11 20:47:51 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:51.187493 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77668f59-6cae-4c3e-8060-061da340c157 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:51.295438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77668f59-6cae-4c3e-8060-061da340c157 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:51.331078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67cbe64-5f98-4e55-9ac2-6a0dc42b9373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:51.331868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cbe64-5f98-4e55-9ac2-6a0dc42b9373 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:51.332150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cbe64-5f98-4e55-9ac2-6a0dc42b9373 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:51.332454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cbe64-5f98-4e55-9ac2-6a0dc42b9373 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:51.333172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67cbe64-5f98-4e55-9ac2-6a0dc42b9373 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:51.333481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67cbe64-5f98-4e55-9ac2-6a0dc42b9373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:51.334868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67cbe64-5f98-4e55-9ac2-6a0dc42b9373 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:51.339517 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 650/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Jul 11 20:47:51 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:53.362075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e672b-6801-4c73-9671-dfd299dd8080 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:53.363091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e672b-6801-4c73-9671-dfd299dd8080 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:53.363091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e672b-6801-4c73-9671-dfd299dd8080 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:53.364016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 643/1293] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:53.370383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4289d-482a-42f3-b969-d04fb763c7a8 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:53.371152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4289d-482a-42f3-b969-d04fb763c7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:53.371282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4289d-482a-42f3-b969-d04fb763c7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:53.398868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-08f4289d-482a-42f3-b969-d04fb763c7a8 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:53.400074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-08f4289d-482a-42f3-b969-d04fb763c7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:53.433199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:53.473966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 651/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:47:53.497826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e672b-6801-4c73-9671-dfd299dd8080 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:53.533453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:53.621483 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e672b-6801-4c73-9671-dfd299dd8080 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:53.657449 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbe12e-f82f-4bfc-9f13-8c7c25f685a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:53.658115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbe12e-f82f-4bfc-9f13-8c7c25f685a3 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:53.658395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbe12e-f82f-4bfc-9f13-8c7c25f685a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:53.658681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbe12e-f82f-4bfc-9f13-8c7c25f685a3 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:53.659928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbe12e-f82f-4bfc-9f13-8c7c25f685a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:53.661350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbe12e-f82f-4bfc-9f13-8c7c25f685a3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:53.679470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 644/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jul 11 20:47:53 2022] POST /identity/v3/services => generated 246 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:47:55.670446 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b7591-557e-4a81-a266-5f214e8db120 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:55.671352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b7591-557e-4a81-a266-5f214e8db120 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:55.671352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b7591-557e-4a81-a266-5f214e8db120 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:55.673568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 652/1296] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:55.679653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc75b81-3cf3-49e1-a22a-36e2e728b278 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:55.680080 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc75b81-3cf3-49e1-a22a-36e2e728b278 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:55.680221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc75b81-3cf3-49e1-a22a-36e2e728b278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:55.699675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7bc75b81-3cf3-49e1-a22a-36e2e728b278 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:55.700917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7bc75b81-3cf3-49e1-a22a-36e2e728b278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:55.781809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 645/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:55 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:47:55.805366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b7591-557e-4a81-a266-5f214e8db120 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:55.935580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b7591-557e-4a81-a266-5f214e8db120 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:55.973478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25e069-e948-486e-b668-a70f92987fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:55.974274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25e069-e948-486e-b668-a70f92987fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:55.974554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25e069-e948-486e-b668-a70f92987fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:55.974809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25e069-e948-486e-b668-a70f92987fa7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:55.975459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25e069-e948-486e-b668-a70f92987fa7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:55.976871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25e069-e948-486e-b668-a70f92987fa7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ee25e069-e948-486e-b668-a70f92987fa7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:55.988932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:55.990049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:55.990908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 653/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jul 11 20:47:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:55.996493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc75b81-3cf3-49e1-a22a-36e2e728b278 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:56.015815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:56.115885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc75b81-3cf3-49e1-a22a-36e2e728b278 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:56.151452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06878496-1660-46a4-b041-761a33c998b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:56.152142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06878496-1660-46a4-b041-761a33c998b8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:56.152421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06878496-1660-46a4-b041-761a33c998b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:56.152694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06878496-1660-46a4-b041-761a33c998b8 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:56.153408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06878496-1660-46a4-b041-761a33c998b8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:56.153707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06878496-1660-46a4-b041-761a33c998b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:56.155092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06878496-1660-46a4-b041-761a33c998b8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:56.160534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 646/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Jul 11 20:47:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:56.166718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25e069-e948-486e-b668-a70f92987fa7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:56.200225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:56.291640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25e069-e948-486e-b668-a70f92987fa7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:56.334354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06cff0-0c7b-4db1-b291-6465791a417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:56.335013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06cff0-0c7b-4db1-b291-6465791a417a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:56.335294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06cff0-0c7b-4db1-b291-6465791a417a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:56.335566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06cff0-0c7b-4db1-b291-6465791a417a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:56.336474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06cff0-0c7b-4db1-b291-6465791a417a None admin] RBAC: Adding query filter params (interface=public, service_id=47ce20c603a2438da30564375c546b7a, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:56.336786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06cff0-0c7b-4db1-b291-6465791a417a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:56.338180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06cff0-0c7b-4db1-b291-6465791a417a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:56.343455 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 654/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jul 11 20:47:56 2022] GET /identity/v3/endpoints?service_id=47ce20c603a2438da30564375c546b7a&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:47:56.352389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06878496-1660-46a4-b041-761a33c998b8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:56.441850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:56.462334 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06878496-1660-46a4-b041-761a33c998b8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:56.498715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da68a7-6faa-49a8-b926-b80fd690dd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:56.499383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da68a7-6faa-49a8-b926-b80fd690dd52 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:56.499662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da68a7-6faa-49a8-b926-b80fd690dd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:56.499942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da68a7-6faa-49a8-b926-b80fd690dd52 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:56.500666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da68a7-6faa-49a8-b926-b80fd690dd52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:56.502043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da68a7-6faa-49a8-b926-b80fd690dd52 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:56.509578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 647/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jul 11 20:47:56 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:47:58.556697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61192560-f166-4ca7-aaa8-b479c023ca37 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:58.557297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61192560-f166-4ca7-aaa8-b479c023ca37 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:58.557297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61192560-f166-4ca7-aaa8-b479c023ca37 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:58.558628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 655/1302] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:47:58.564924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c376e77-761e-40c0-9b7d-b408cc5e3a09 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:58.565543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c376e77-761e-40c0-9b7d-b408cc5e3a09 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:58.566004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c376e77-761e-40c0-9b7d-b408cc5e3a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:58.586293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0c376e77-761e-40c0-9b7d-b408cc5e3a09 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:47:58.587512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0c376e77-761e-40c0-9b7d-b408cc5e3a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:47:58.657839 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 648/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:47:58 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 20:47:58.692961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61192560-f166-4ca7-aaa8-b479c023ca37 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:58.813409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:58.826827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61192560-f166-4ca7-aaa8-b479c023ca37 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:58.863524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07817c8-390a-4dec-8cc2-2a7f445509fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:58.864354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07817c8-390a-4dec-8cc2-2a7f445509fe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:58.864629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07817c8-390a-4dec-8cc2-2a7f445509fe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:58.864898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07817c8-390a-4dec-8cc2-2a7f445509fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:58.865545 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07817c8-390a-4dec-8cc2-2a7f445509fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:58.866909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07817c8-390a-4dec-8cc2-2a7f445509fe None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d07817c8-390a-4dec-8cc2-2a7f445509fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 20:47:58.879012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 20:47:58.880144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 11 20:47:58.880994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 656/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jul 11 20:47:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 20:47:58.886446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c376e77-761e-40c0-9b7d-b408cc5e3a09 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:58.964678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:47:58.988984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c376e77-761e-40c0-9b7d-b408cc5e3a09 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:59.024683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af5f89-fcf4-48a9-8571-76607444e15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:59.025339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af5f89-fcf4-48a9-8571-76607444e15f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:59.025617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af5f89-fcf4-48a9-8571-76607444e15f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:59.025885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af5f89-fcf4-48a9-8571-76607444e15f None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:59.026589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af5f89-fcf4-48a9-8571-76607444e15f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:47:59.026890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af5f89-fcf4-48a9-8571-76607444e15f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:59.028291 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af5f89-fcf4-48a9-8571-76607444e15f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:59.034035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 649/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Jul 11 20:47:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:47:59.043410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07817c8-390a-4dec-8cc2-2a7f445509fe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:47:59.163506 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07817c8-390a-4dec-8cc2-2a7f445509fe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:47:59.200699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270df091-a735-46e5-95d3-0a980f05f6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:47:59.201352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df091-a735-46e5-95d3-0a980f05f6c1 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:47:59.201630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df091-a735-46e5-95d3-0a980f05f6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:47:59.201899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df091-a735-46e5-95d3-0a980f05f6c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:47:59.202633 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270df091-a735-46e5-95d3-0a980f05f6c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:47:59.204008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270df091-a735-46e5-95d3-0a980f05f6c1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:47:59.226037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 657/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jul 11 20:47:59 2022] POST /identity/v3/endpoints => generated 331 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:48:09.030555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=30 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:48:09.221695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:02.705015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa610e3-a8d1-477a-9465-fed8408ba58f None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:02.706223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa610e3-a8d1-477a-9465-fed8408ba58f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:02.706727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa610e3-a8d1-477a-9465-fed8408ba58f None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:02.709011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 650/1307] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:02.714069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4b2ca-151a-42c4-a07e-a9ddf5654b46 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:02.714534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4b2ca-151a-42c4-a07e-a9ddf5654b46 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:02.714614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4b2ca-151a-42c4-a07e-a9ddf5654b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:02.796464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-23a4b2ca-151a-42c4-a07e-a9ddf5654b46 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:02.797583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-23a4b2ca-151a-42c4-a07e-a9ddf5654b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:02.946049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 658/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:02.995394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59316c-5185-40e1-b4b6-b84738dd5612 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:02.995977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59316c-5185-40e1-b4b6-b84738dd5612 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:02.996379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59316c-5185-40e1-b4b6-b84738dd5612 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:02.998059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 651/1309] 10.222.0.20 () {54 vars in 964 bytes} [Mon Jul 11 20:49:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:03.007179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976344-0f31-4d8f-b30a-78365ad35239 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:03.007473 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976344-0f31-4d8f-b30a-78365ad35239 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:03.007749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976344-0f31-4d8f-b30a-78365ad35239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:03.109097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1d976344-0f31-4d8f-b30a-78365ad35239 None None] MFA Rules not processed for user `0574d5c8974f43218653208ca72c43bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:03.110610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1d976344-0f31-4d8f-b30a-78365ad35239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:03.139930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:03.207804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 659/1310] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Jul 11 20:49:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:03.216657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59316c-5185-40e1-b4b6-b84738dd5612 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:03.218705 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6d59316c-5185-40e1-b4b6-b84738dd5612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:03.330953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59316c-5185-40e1-b4b6-b84738dd5612 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:03.411333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c3a76-65af-4d66-af27-0a714978da40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:03.412358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3a76-65af-4d66-af27-0a714978da40 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:03.412822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3a76-65af-4d66-af27-0a714978da40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:03.413178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3a76-65af-4d66-af27-0a714978da40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:03.414963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e23c3a76-65af-4d66-af27-0a714978da40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:03.473332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:03.532198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c3a76-65af-4d66-af27-0a714978da40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:03.533834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c3a76-65af-4d66-af27-0a714978da40 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:03.553940 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 652/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:05.838924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce86212-969c-40a1-ac97-4d77f892498e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:05.839385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce86212-969c-40a1-ac97-4d77f892498e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:05.839480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce86212-969c-40a1-ac97-4d77f892498e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:05.840893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 660/1312] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:05.846230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9170cd-68ef-4b9a-9f64-716f219c0d48 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:05.846665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9170cd-68ef-4b9a-9f64-716f219c0d48 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:05.846894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9170cd-68ef-4b9a-9f64-716f219c0d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:05.918588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9c9170cd-68ef-4b9a-9f64-716f219c0d48 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:05.919793 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9c9170cd-68ef-4b9a-9f64-716f219c0d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:06.019012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 653/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:06.057766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce86212-969c-40a1-ac97-4d77f892498e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:06.059557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9ce86212-969c-40a1-ac97-4d77f892498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:06.184436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce86212-969c-40a1-ac97-4d77f892498e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:06.261652 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:06.264877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec9b59-98d0-43ff-a971-1a1fa0979dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:06.265662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec9b59-98d0-43ff-a971-1a1fa0979dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:06.266105 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec9b59-98d0-43ff-a971-1a1fa0979dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:06.266428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec9b59-98d0-43ff-a971-1a1fa0979dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:06.269288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6fec9b59-98d0-43ff-a971-1a1fa0979dc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:06.389782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec9b59-98d0-43ff-a971-1a1fa0979dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:06.391253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec9b59-98d0-43ff-a971-1a1fa0979dc9 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:06.410840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 661/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:08.705266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bf0b8-1390-4a8c-93ab-a35e6664a8db None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:08.705798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bf0b8-1390-4a8c-93ab-a35e6664a8db None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:08.705798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bf0b8-1390-4a8c-93ab-a35e6664a8db None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:08.707124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 654/1315] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:08.713678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbd5dc-c2f7-4b91-b05e-a6f87a5c2339 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:08.714408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbd5dc-c2f7-4b91-b05e-a6f87a5c2339 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:08.714552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbd5dc-c2f7-4b91-b05e-a6f87a5c2339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:08.783756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9ecbd5dc-c2f7-4b91-b05e-a6f87a5c2339 None None] MFA Rules not processed for user `0c0d3071ebef4f658f2da266190c1597`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:08.785014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9ecbd5dc-c2f7-4b91-b05e-a6f87a5c2339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:08.842158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:08.886913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 662/1316] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:08.930020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bf0b8-1390-4a8c-93ab-a35e6664a8db None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:08.932561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c32bf0b8-1390-4a8c-93ab-a35e6664a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:08.947355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:09.045085 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bf0b8-1390-4a8c-93ab-a35e6664a8db None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:09.120001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a223bb5-2809-48fa-b422-ca6dc72b8a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:09.120678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a223bb5-2809-48fa-b422-ca6dc72b8a2e service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:09.120959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a223bb5-2809-48fa-b422-ca6dc72b8a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:09.121232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a223bb5-2809-48fa-b422-ca6dc72b8a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:09.122876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2a223bb5-2809-48fa-b422-ca6dc72b8a2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:09.196488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:09.251283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a223bb5-2809-48fa-b422-ca6dc72b8a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:09.252767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a223bb5-2809-48fa-b422-ca6dc72b8a2e service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:09.272298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 655/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:08 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:11.851093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca986c9-cee4-4f29-aeca-b8fd65c8f8e9 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:11.851586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca986c9-cee4-4f29-aeca-b8fd65c8f8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:11.851586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca986c9-cee4-4f29-aeca-b8fd65c8f8e9 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:11.853057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 663/1318] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:11.858413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b7683-87a2-4839-bde3-6be48765dfb6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:11.858859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b7683-87a2-4839-bde3-6be48765dfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:11.858952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b7683-87a2-4839-bde3-6be48765dfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:11.930534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-da8b7683-87a2-4839-bde3-6be48765dfb6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:11.931791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-da8b7683-87a2-4839-bde3-6be48765dfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:12.034336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 656/1319] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:12.072704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca986c9-cee4-4f29-aeca-b8fd65c8f8e9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:12.074247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7ca986c9-cee4-4f29-aeca-b8fd65c8f8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:12.194035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca986c9-cee4-4f29-aeca-b8fd65c8f8e9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:12.270157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df0025-c05b-40f0-af26-bc674b1c80b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:12.270798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df0025-c05b-40f0-af26-bc674b1c80b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:12.271074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df0025-c05b-40f0-af26-bc674b1c80b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:12.271349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df0025-c05b-40f0-af26-bc674b1c80b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:12.273285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-88df0025-c05b-40f0-af26-bc674b1c80b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:12.280935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:12.394771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df0025-c05b-40f0-af26-bc674b1c80b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:12.396247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df0025-c05b-40f0-af26-bc674b1c80b9 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:12.414615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 664/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:15.126141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6bff-24d4-46ef-a2e5-c54c103fe12a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:15.126665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6bff-24d4-46ef-a2e5-c54c103fe12a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:15.126778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6bff-24d4-46ef-a2e5-c54c103fe12a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:15.128212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 657/1321] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:15.133719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fecfed-e1f4-481e-a54d-7fdd837b8997 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:15.134111 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fecfed-e1f4-481e-a54d-7fdd837b8997 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:15.134329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fecfed-e1f4-481e-a54d-7fdd837b8997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:15.202585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c8fecfed-e1f4-481e-a54d-7fdd837b8997 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:15.203786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c8fecfed-e1f4-481e-a54d-7fdd837b8997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:15.263975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:15.321056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 665/1322] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:15.358333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a6bff-24d4-46ef-a2e5-c54c103fe12a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:15.359899 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b41a6bff-24d4-46ef-a2e5-c54c103fe12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:15.381934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:15.474916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a6bff-24d4-46ef-a2e5-c54c103fe12a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:15.557877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2ba64-470d-4f9d-9d30-34374db12191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:15.558534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2ba64-470d-4f9d-9d30-34374db12191 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:15.558809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2ba64-470d-4f9d-9d30-34374db12191 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:15.559078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2ba64-470d-4f9d-9d30-34374db12191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:15.560737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7eb2ba64-470d-4f9d-9d30-34374db12191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:15.635105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:15.674163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2ba64-470d-4f9d-9d30-34374db12191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:15.675607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2ba64-470d-4f9d-9d30-34374db12191 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:15.693947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 658/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:17.805919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee026a-c14b-4995-abcb-d933941f6bf9 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:17.806463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee026a-c14b-4995-abcb-d933941f6bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:17.806463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee026a-c14b-4995-abcb-d933941f6bf9 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:17.807738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 666/1324] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:17.813225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf2e2b-10c5-4c16-b7ec-81700aa21e86 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:17.813666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf2e2b-10c5-4c16-b7ec-81700aa21e86 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:17.813796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf2e2b-10c5-4c16-b7ec-81700aa21e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:17.882907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-54bf2e2b-10c5-4c16-b7ec-81700aa21e86 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:17.884092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-54bf2e2b-10c5-4c16-b7ec-81700aa21e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:17.983403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 659/1325] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:18.033389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6002c-9ec6-40a7-bea1-b76b866866c6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:18.033673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6002c-9ec6-40a7-bea1-b76b866866c6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:18.033942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6002c-9ec6-40a7-bea1-b76b866866c6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:18.035359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 667/1326] 10.222.0.20 () {54 vars in 964 bytes} [Mon Jul 11 20:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:18.042258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c0381-0f05-4775-a43f-205413a83690 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:18.042542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c0381-0f05-4775-a43f-205413a83690 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:18.042812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c0381-0f05-4775-a43f-205413a83690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:18.075312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:18.121978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-313c0381-0f05-4775-a43f-205413a83690 None None] MFA Rules not processed for user `0574d5c8974f43218653208ca72c43bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:18.123169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-313c0381-0f05-4775-a43f-205413a83690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:18.206152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 660/1327] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Jul 11 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:18.216281 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6002c-9ec6-40a7-bea1-b76b866866c6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:18.217852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d7d6002c-9ec6-40a7-bea1-b76b866866c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:18.326614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6002c-9ec6-40a7-bea1-b76b866866c6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:18.412753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f286ac9-1e52-4a0c-9e99-43600920da46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:18.413430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f286ac9-1e52-4a0c-9e99-43600920da46 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:18.413710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f286ac9-1e52-4a0c-9e99-43600920da46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:18.413978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f286ac9-1e52-4a0c-9e99-43600920da46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:18.415662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6f286ac9-1e52-4a0c-9e99-43600920da46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:18.434056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:18.532013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f286ac9-1e52-4a0c-9e99-43600920da46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:18.533511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f286ac9-1e52-4a0c-9e99-43600920da46 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:18.552797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 668/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:20.828171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66ac44-4929-4973-aeba-8c1b810dbefa None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:20.828769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66ac44-4929-4973-aeba-8c1b810dbefa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:20.828769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66ac44-4929-4973-aeba-8c1b810dbefa None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:20.830360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 661/1329] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:20.835883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3e2cd-7fe5-4e9d-97db-cb142c3e7e29 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:20.836407 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3e2cd-7fe5-4e9d-97db-cb142c3e7e29 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:20.836599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3e2cd-7fe5-4e9d-97db-cb142c3e7e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:20.902304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-85b3e2cd-7fe5-4e9d-97db-cb142c3e7e29 None None] MFA Rules not processed for user `0c0d3071ebef4f658f2da266190c1597`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:20.903505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-85b3e2cd-7fe5-4e9d-97db-cb142c3e7e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:20.990552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:21.027042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 669/1330] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:21.065293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66ac44-4929-4973-aeba-8c1b810dbefa None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:21.067185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fa66ac44-4929-4973-aeba-8c1b810dbefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:21.184355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66ac44-4929-4973-aeba-8c1b810dbefa None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:21.206024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:21.262484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac09d6a-6ebb-4723-8e6c-4d8200c7f22e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:21.263150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac09d6a-6ebb-4723-8e6c-4d8200c7f22e service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:21.263428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac09d6a-6ebb-4723-8e6c-4d8200c7f22e service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:21.263696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac09d6a-6ebb-4723-8e6c-4d8200c7f22e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:21.265393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-eac09d6a-6ebb-4723-8e6c-4d8200c7f22e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:21.388212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac09d6a-6ebb-4723-8e6c-4d8200c7f22e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:21.389667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac09d6a-6ebb-4723-8e6c-4d8200c7f22e service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:21.410206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 662/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:23.813158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707bdd0-1f84-4b2c-89d4-6bfdf963d01c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:23.814147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707bdd0-1f84-4b2c-89d4-6bfdf963d01c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:23.814147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707bdd0-1f84-4b2c-89d4-6bfdf963d01c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:23.816817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 670/1332] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:23.822129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c0856-cbe5-4d38-8e96-a68a82725a0e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:23.822608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c0856-cbe5-4d38-8e96-a68a82725a0e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:23.822718 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c0856-cbe5-4d38-8e96-a68a82725a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:23.899307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:23.902556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4d9c0856-cbe5-4d38-8e96-a68a82725a0e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:23.905220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4d9c0856-cbe5-4d38-8e96-a68a82725a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:24.009505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 663/1333] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:24.050217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707bdd0-1f84-4b2c-89d4-6bfdf963d01c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:24.052724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4707bdd0-1f84-4b2c-89d4-6bfdf963d01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:24.170286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707bdd0-1f84-4b2c-89d4-6bfdf963d01c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:24.267207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226d87a-3d52-4f67-931a-4f659d9d9cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:24.267866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226d87a-3d52-4f67-931a-4f659d9d9cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:24.268141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226d87a-3d52-4f67-931a-4f659d9d9cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:24.268445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226d87a-3d52-4f67-931a-4f659d9d9cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:24.270104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9226d87a-3d52-4f67-931a-4f659d9d9cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:24.297712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:24.383714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9226d87a-3d52-4f67-931a-4f659d9d9cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:24.385190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9226d87a-3d52-4f67-931a-4f659d9d9cd1 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:24.403786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 671/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:26.498140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fbd51-50a0-4f2b-8105-9db920759459 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:26.498704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fbd51-50a0-4f2b-8105-9db920759459 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:26.498704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fbd51-50a0-4f2b-8105-9db920759459 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:26.500078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 664/1335] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:26.506374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff73f76-2b1f-424e-a2d6-c61f4eb32317 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:26.507375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff73f76-2b1f-424e-a2d6-c61f4eb32317 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:26.507626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff73f76-2b1f-424e-a2d6-c61f4eb32317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:26.583036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fff73f76-2b1f-424e-a2d6-c61f4eb32317 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:26.584298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fff73f76-2b1f-424e-a2d6-c61f4eb32317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:26.663326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:26.688091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 672/1336] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:26.740451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53fbd51-50a0-4f2b-8105-9db920759459 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:26.741999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b53fbd51-50a0-4f2b-8105-9db920759459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:26.854554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53fbd51-50a0-4f2b-8105-9db920759459 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:26.885389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:26.943176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac248d-cd10-4c71-b10c-8da29f82a79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:26.943824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac248d-cd10-4c71-b10c-8da29f82a79a service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:26.944101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac248d-cd10-4c71-b10c-8da29f82a79a service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:26.944445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac248d-cd10-4c71-b10c-8da29f82a79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:26.946225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-69ac248d-cd10-4c71-b10c-8da29f82a79a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:27.060753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac248d-cd10-4c71-b10c-8da29f82a79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:27.062112 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac248d-cd10-4c71-b10c-8da29f82a79a service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:27.080538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 665/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:29.446847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455e13b-6d38-4849-a51d-91131df6e995 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:29.447741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455e13b-6d38-4849-a51d-91131df6e995 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:29.447741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455e13b-6d38-4849-a51d-91131df6e995 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:29.449988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 673/1338] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:29.456586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f140d1-35f7-44d9-8297-d7f4a6d7b326 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:29.457024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f140d1-35f7-44d9-8297-d7f4a6d7b326 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:29.457140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f140d1-35f7-44d9-8297-d7f4a6d7b326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:29.544737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-77f140d1-35f7-44d9-8297-d7f4a6d7b326 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:29.546273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-77f140d1-35f7-44d9-8297-d7f4a6d7b326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:29.558049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:29.654428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 666/1339] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:29.690176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455e13b-6d38-4849-a51d-91131df6e995 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:29.692123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b455e13b-6d38-4849-a51d-91131df6e995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:29.798881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455e13b-6d38-4849-a51d-91131df6e995 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:29.876500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f5317-d0a0-453b-bb84-368b20310dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:29.877156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f5317-d0a0-453b-bb84-368b20310dbd service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:29.877431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f5317-d0a0-453b-bb84-368b20310dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:29.877701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f5317-d0a0-453b-bb84-368b20310dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:29.879340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eb8f5317-d0a0-453b-bb84-368b20310dbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:29.918362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:30.005313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f5317-d0a0-453b-bb84-368b20310dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:30.006761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f5317-d0a0-453b-bb84-368b20310dbd service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:30.026352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 674/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:49:34.129260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00c3d2-2518-418c-9e52-0e637ffb1512 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:34.129836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00c3d2-2518-418c-9e52-0e637ffb1512 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:34.129836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00c3d2-2518-418c-9e52-0e637ffb1512 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:34.131266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 667/1341] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:34.136638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3211417-6b50-4755-8285-78c136c808b6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:34.137052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3211417-6b50-4755-8285-78c136c808b6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:34.137299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3211417-6b50-4755-8285-78c136c808b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:34.204943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a3211417-6b50-4755-8285-78c136c808b6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:34.206077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a3211417-6b50-4755-8285-78c136c808b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:34.297432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:34.309811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 675/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:34.348791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00c3d2-2518-418c-9e52-0e637ffb1512 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:34.350305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0c00c3d2-2518-418c-9e52-0e637ffb1512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:34.462720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00c3d2-2518-418c-9e52-0e637ffb1512 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:34.503416 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:34.543192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ec034-34f1-4f75-a23b-6219e11d75cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:34.543912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ec034-34f1-4f75-a23b-6219e11d75cc service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:34.544324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ec034-34f1-4f75-a23b-6219e11d75cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:34.544600 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ec034-34f1-4f75-a23b-6219e11d75cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:34.546317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1f2ec034-34f1-4f75-a23b-6219e11d75cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:34.659623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ec034-34f1-4f75-a23b-6219e11d75cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:34.661328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ec034-34f1-4f75-a23b-6219e11d75cc service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:34.679747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 668/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:37.393282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09e382-d727-48c8-9948-978f5d38f101 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:37.393886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09e382-d727-48c8-9948-978f5d38f101 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:37.393886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09e382-d727-48c8-9948-978f5d38f101 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:37.395249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 676/1344] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:37.402076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05210b2-f50e-475f-9e5f-1017d47842d9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:37.402811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05210b2-f50e-475f-9e5f-1017d47842d9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:37.402943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05210b2-f50e-475f-9e5f-1017d47842d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:37.479616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f05210b2-f50e-475f-9e5f-1017d47842d9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:37.480871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f05210b2-f50e-475f-9e5f-1017d47842d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:37.499610 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:37.594690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 669/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:37.631244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09e382-d727-48c8-9948-978f5d38f101 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:37.632852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2d09e382-d727-48c8-9948-978f5d38f101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:37.740470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09e382-d727-48c8-9948-978f5d38f101 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:37.819636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da42f3-be9a-4c82-a649-e6feec5a1a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:37.820421 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da42f3-be9a-4c82-a649-e6feec5a1a64 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:37.820770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da42f3-be9a-4c82-a649-e6feec5a1a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:37.821048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da42f3-be9a-4c82-a649-e6feec5a1a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:37.822892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-56da42f3-be9a-4c82-a649-e6feec5a1a64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:37.868899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:37.936705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da42f3-be9a-4c82-a649-e6feec5a1a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:37.938167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da42f3-be9a-4c82-a649-e6feec5a1a64 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:37.957866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 677/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:49:41.981623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776d68b-ff1c-4f27-b62e-8bc872d25169 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:41.982188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776d68b-ff1c-4f27-b62e-8bc872d25169 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:41.982188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776d68b-ff1c-4f27-b62e-8bc872d25169 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:41.983725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 670/1347] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:41.988863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee4ff7-1a91-441f-89d6-1b36bcc6d578 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:41.989265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee4ff7-1a91-441f-89d6-1b36bcc6d578 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:41.989422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee4ff7-1a91-441f-89d6-1b36bcc6d578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:42.058570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6fee4ff7-1a91-441f-89d6-1b36bcc6d578 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:42.059887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6fee4ff7-1a91-441f-89d6-1b36bcc6d578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:42.175614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:42.178899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 678/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:42.218779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776d68b-ff1c-4f27-b62e-8bc872d25169 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:42.220402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d776d68b-ff1c-4f27-b62e-8bc872d25169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:42.341592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776d68b-ff1c-4f27-b62e-8bc872d25169 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:42.392036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:42.421107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f7619-d956-4f6b-ab92-871a4d50714e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:42.421771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f7619-d956-4f6b-ab92-871a4d50714e service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:42.422054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f7619-d956-4f6b-ab92-871a4d50714e service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:42.422326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f7619-d956-4f6b-ab92-871a4d50714e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:42.423987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8a0f7619-d956-4f6b-ab92-871a4d50714e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:42.549155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f7619-d956-4f6b-ab92-871a4d50714e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:42.550662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f7619-d956-4f6b-ab92-871a4d50714e service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:42.572612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 671/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:49:47.523959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d146ee-992c-456a-ade8-b97b0e00dc96 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:47.524657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d146ee-992c-456a-ade8-b97b0e00dc96 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:47.524657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d146ee-992c-456a-ade8-b97b0e00dc96 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:47.526167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 679/1350] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 20:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:47.542009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66bc2e-3321-4418-9ffa-7a73a0bbb905 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:47.542894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66bc2e-3321-4418-9ffa-7a73a0bbb905 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:47.543390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66bc2e-3321-4418-9ffa-7a73a0bbb905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:47.626798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bb66bc2e-3321-4418-9ffa-7a73a0bbb905 None None] MFA Rules not processed for user `946d11b552754364b55eb495f68ab852`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:47.628423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bb66bc2e-3321-4418-9ffa-7a73a0bbb905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:47.653050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:47.752471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 672/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 20:49:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:51.837653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549bd5b-51ab-48b2-928a-d687530eebfa None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:51.838192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549bd5b-51ab-48b2-928a-d687530eebfa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:51.838192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549bd5b-51ab-48b2-928a-d687530eebfa None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:51.839552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 680/1352] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 20:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:51.844951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5feaee-deb6-4cf4-9ec6-7dbabac77eff None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:51.845332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5feaee-deb6-4cf4-9ec6-7dbabac77eff None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:51.845332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5feaee-deb6-4cf4-9ec6-7dbabac77eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:51.927835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cb5feaee-deb6-4cf4-9ec6-7dbabac77eff None None] MFA Rules not processed for user `946d11b552754364b55eb495f68ab852`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:51.929081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cb5feaee-deb6-4cf4-9ec6-7dbabac77eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:52.005372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:52.049982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 673/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 20:49:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:53.384052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3eb4f-b7e4-4dbe-8e6d-c454eb75520f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:53.384683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3eb4f-b7e4-4dbe-8e6d-c454eb75520f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:53.384683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3eb4f-b7e4-4dbe-8e6d-c454eb75520f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:53.386011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 681/1354] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 20:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:53.391000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38c0c-8c75-4463-ac61-a6c63811ea55 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:53.391452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38c0c-8c75-4463-ac61-a6c63811ea55 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:53.391550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38c0c-8c75-4463-ac61-a6c63811ea55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:53.470848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bbf38c0c-8c75-4463-ac61-a6c63811ea55 None None] MFA Rules not processed for user `946d11b552754364b55eb495f68ab852`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:53.472045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bbf38c0c-8c75-4463-ac61-a6c63811ea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:53.581815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 674/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 20:49:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:56.973542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961aa83-6d30-4d43-bf6d-936d6fbdb97c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:56.974088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961aa83-6d30-4d43-bf6d-936d6fbdb97c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:56.974088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961aa83-6d30-4d43-bf6d-936d6fbdb97c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:56.975350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 682/1356] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:56.980841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc178d33-1fb8-4ccf-97d4-83e096e8a063 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:56.981313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc178d33-1fb8-4ccf-97d4-83e096e8a063 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:56.981418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc178d33-1fb8-4ccf-97d4-83e096e8a063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:57.018926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:57.053318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cc178d33-1fb8-4ccf-97d4-83e096e8a063 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:57.054662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cc178d33-1fb8-4ccf-97d4-83e096e8a063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:57.158464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 675/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:57.213500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf82d6-2a13-4e7c-a1f0-a1bc59ea68bf None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:57.213983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf82d6-2a13-4e7c-a1f0-a1bc59ea68bf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:57.214422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf82d6-2a13-4e7c-a1f0-a1bc59ea68bf None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:57.216459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 683/1358] 10.222.0.20 () {54 vars in 961 bytes} [Mon Jul 11 20:49:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:49:57.223138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b7f4e-bcdc-484f-a5f8-acba81115326 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:57.223442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b7f4e-bcdc-484f-a5f8-acba81115326 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:57.223720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b7f4e-bcdc-484f-a5f8-acba81115326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:57.305135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-139b7f4e-bcdc-484f-a5f8-acba81115326 None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:49:57.306490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-139b7f4e-bcdc-484f-a5f8-acba81115326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:57.359493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:57.432432 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 676/1359] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Jul 11 20:49:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:49:57.440269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf82d6-2a13-4e7c-a1f0-a1bc59ea68bf None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:49:57.442226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c4bf82d6-2a13-4e7c-a1f0-a1bc59ea68bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:57.603573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf82d6-2a13-4e7c-a1f0-a1bc59ea68bf None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:49:57.680128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:49:57.723500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbcc77-8aee-45de-8d12-d3bc3325fdbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:49:57.724180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbcc77-8aee-45de-8d12-d3bc3325fdbd service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:49:57.724468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbcc77-8aee-45de-8d12-d3bc3325fdbd service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:49:57.724763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbcc77-8aee-45de-8d12-d3bc3325fdbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:49:57.726440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-69cbcc77-8aee-45de-8d12-d3bc3325fdbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:49:57.822777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbcc77-8aee-45de-8d12-d3bc3325fdbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:49:57.824537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbcc77-8aee-45de-8d12-d3bc3325fdbd service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:49:57.847917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 684/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:00.093237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559f898-f13b-4c7b-84d6-d08ff9238d13 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:00.094129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559f898-f13b-4c7b-84d6-d08ff9238d13 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:00.094408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559f898-f13b-4c7b-84d6-d08ff9238d13 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:00.095843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 677/1361] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:00.101510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea7b8c-9b96-4ec7-9131-672842e19d25 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:00.102037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea7b8c-9b96-4ec7-9131-672842e19d25 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:00.102281 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea7b8c-9b96-4ec7-9131-672842e19d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:00.170356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b4ea7b8c-9b96-4ec7-9131-672842e19d25 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:00.171743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b4ea7b8c-9b96-4ec7-9131-672842e19d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:00.211952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:00.276667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 685/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:00.324577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559f898-f13b-4c7b-84d6-d08ff9238d13 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:00.326164 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e559f898-f13b-4c7b-84d6-d08ff9238d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:00.517077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559f898-f13b-4c7b-84d6-d08ff9238d13 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:00.527105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:00.668987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbda70-2d21-4504-98c2-90b449d271f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:00.669725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbda70-2d21-4504-98c2-90b449d271f9 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:00.670068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbda70-2d21-4504-98c2-90b449d271f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:00.670402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbda70-2d21-4504-98c2-90b449d271f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:00.672169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f5fbda70-2d21-4504-98c2-90b449d271f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:00.768626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbda70-2d21-4504-98c2-90b449d271f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:00.770792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbda70-2d21-4504-98c2-90b449d271f9 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:00.790306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 678/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:03.168554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230ffa-3e24-496c-9bd9-7edf9ecfd5fe None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:03.169432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230ffa-3e24-496c-9bd9-7edf9ecfd5fe None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:03.169845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230ffa-3e24-496c-9bd9-7edf9ecfd5fe None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:03.172875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 686/1364] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:03.180456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741506a-58af-494f-8631-13990c6f126a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:03.181299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741506a-58af-494f-8631-13990c6f126a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:03.181769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741506a-58af-494f-8631-13990c6f126a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:03.200171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:03.274056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9741506a-58af-494f-8631-13990c6f126a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:03.275372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9741506a-58af-494f-8631-13990c6f126a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:03.384172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 679/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:03.430596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a4e42-2d7d-4800-a921-0df3a17286eb None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:03.430868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a4e42-2d7d-4800-a921-0df3a17286eb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:03.431661 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a4e42-2d7d-4800-a921-0df3a17286eb None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:03.433065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 687/1366] 10.222.0.20 () {54 vars in 961 bytes} [Mon Jul 11 20:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:03.438795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3e6f-32ea-4ef8-bc63-2e600b191e32 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:03.439318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3e6f-32ea-4ef8-bc63-2e600b191e32 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:03.439703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3e6f-32ea-4ef8-bc63-2e600b191e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:03.521093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-dcef3e6f-32ea-4ef8-bc63-2e600b191e32 None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:03.522334 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dcef3e6f-32ea-4ef8-bc63-2e600b191e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:03.561735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:03.656517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 680/1367] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Jul 11 20:50:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:03.664636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a4e42-2d7d-4800-a921-0df3a17286eb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:03.666325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ac3a4e42-2d7d-4800-a921-0df3a17286eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:03.822861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a4e42-2d7d-4800-a921-0df3a17286eb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:03.846987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:03.966769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da71d88-80e6-4cba-8ef9-9908ae682fcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:03.967539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da71d88-80e6-4cba-8ef9-9908ae682fcf service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:03.967920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da71d88-80e6-4cba-8ef9-9908ae682fcf service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:03.968359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da71d88-80e6-4cba-8ef9-9908ae682fcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:03.970123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8da71d88-80e6-4cba-8ef9-9908ae682fcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:04.064503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da71d88-80e6-4cba-8ef9-9908ae682fcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:04.066481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da71d88-80e6-4cba-8ef9-9908ae682fcf service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:04.087239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 688/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:50:05.024877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a020f6-9717-4d08-b63e-1c3655515486 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:05.025420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a020f6-9717-4d08-b63e-1c3655515486 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:05.025420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a020f6-9717-4d08-b63e-1c3655515486 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:05.026927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 681/1369] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:05.032551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a9299-7fa2-4bad-b4ec-36111385a218 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:05.032951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a9299-7fa2-4bad-b4ec-36111385a218 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:05.033096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a9299-7fa2-4bad-b4ec-36111385a218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:05.066697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:05.105228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b48a9299-7fa2-4bad-b4ec-36111385a218 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:05.106439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b48a9299-7fa2-4bad-b4ec-36111385a218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:05.212143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 689/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:05.562486 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66cbb4-3d5f-49c3-b2d8-b19f9ba3fb57 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:05.562486 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66cbb4-3d5f-49c3-b2d8-b19f9ba3fb57 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:05.562486 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66cbb4-3d5f-49c3-b2d8-b19f9ba3fb57 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:05.564531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 682/1371] 10.222.0.20 () {54 vars in 963 bytes} [Mon Jul 11 20:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:05.569427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebfe63-654d-4c82-88e5-843928cb7c52 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:05.569831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebfe63-654d-4c82-88e5-843928cb7c52 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:05.570169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebfe63-654d-4c82-88e5-843928cb7c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:05.650965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c9ebfe63-654d-4c82-88e5-843928cb7c52 None None] MFA Rules not processed for user `dee928cd256446cb9e000ceb6e9ef636`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:05.652222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c9ebfe63-654d-4c82-88e5-843928cb7c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:05.703432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:05.737192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 690/1372] 10.222.0.20 () {58 vars in 1060 bytes} [Mon Jul 11 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:05.745492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66cbb4-3d5f-49c3-b2d8-b19f9ba3fb57 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:05.747008 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bf66cbb4-3d5f-49c3-b2d8-b19f9ba3fb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:05.865317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66cbb4-3d5f-49c3-b2d8-b19f9ba3fb57 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:05.910806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:05.948008 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c943ff-922e-49a9-a49a-73487324a877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:05.948861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c943ff-922e-49a9-a49a-73487324a877 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:05.949238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c943ff-922e-49a9-a49a-73487324a877 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:05.949599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c943ff-922e-49a9-a49a-73487324a877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:05.951408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-43c943ff-922e-49a9-a49a-73487324a877 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:06.066835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c943ff-922e-49a9-a49a-73487324a877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:06.068320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c943ff-922e-49a9-a49a-73487324a877 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:06.090226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 683/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:50:06.221796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8a074-e67a-4ad7-b2b9-96702cea8795 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:06.222242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8a074-e67a-4ad7-b2b9-96702cea8795 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:06.222489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8a074-e67a-4ad7-b2b9-96702cea8795 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:06.224092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 691/1374] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:06.229157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242426be-8c0d-4b0c-90bd-2c7d02279e30 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:06.229506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242426be-8c0d-4b0c-90bd-2c7d02279e30 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:06.229687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242426be-8c0d-4b0c-90bd-2c7d02279e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:06.304528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-242426be-8c0d-4b0c-90bd-2c7d02279e30 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:06.306700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-242426be-8c0d-4b0c-90bd-2c7d02279e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:06.332606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:06.431808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 684/1375] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:06.475327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8a074-e67a-4ad7-b2b9-96702cea8795 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:06.477864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e5d8a074-e67a-4ad7-b2b9-96702cea8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:06.645473 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8a074-e67a-4ad7-b2b9-96702cea8795 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:06.700812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:06.765820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65374dd-1585-4259-9bad-9b3d2d410c6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:06.766474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65374dd-1585-4259-9bad-9b3d2d410c6d service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:06.766766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65374dd-1585-4259-9bad-9b3d2d410c6d service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:06.767030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65374dd-1585-4259-9bad-9b3d2d410c6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:06.769059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e65374dd-1585-4259-9bad-9b3d2d410c6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:06.863828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65374dd-1585-4259-9bad-9b3d2d410c6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:06.865557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65374dd-1585-4259-9bad-9b3d2d410c6d service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:06.884134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 692/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:08.928242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec67f0-8b21-4afd-a06c-eae1588999ed None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:08.928901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec67f0-8b21-4afd-a06c-eae1588999ed None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:08.928901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec67f0-8b21-4afd-a06c-eae1588999ed None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:08.930450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 685/1377] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:08.936005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa59cb-ca25-4af6-bb04-bb83211173c1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:08.936518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa59cb-ca25-4af6-bb04-bb83211173c1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:08.936787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa59cb-ca25-4af6-bb04-bb83211173c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:09.025538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-03fa59cb-ca25-4af6-bb04-bb83211173c1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:09.026744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-03fa59cb-ca25-4af6-bb04-bb83211173c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:09.033555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:09.145929 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 693/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:09.189759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec67f0-8b21-4afd-a06c-eae1588999ed None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:09.191742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-21ec67f0-8b21-4afd-a06c-eae1588999ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:09.348306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec67f0-8b21-4afd-a06c-eae1588999ed None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:09.352780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:09.497137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce18e0a-28fc-4dc7-9887-697aabc4c7f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:09.498083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce18e0a-28fc-4dc7-9887-697aabc4c7f8 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:09.498622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce18e0a-28fc-4dc7-9887-697aabc4c7f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:09.499047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce18e0a-28fc-4dc7-9887-697aabc4c7f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:09.501800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fce18e0a-28fc-4dc7-9887-697aabc4c7f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:09.621079 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce18e0a-28fc-4dc7-9887-697aabc4c7f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:09.623102 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce18e0a-28fc-4dc7-9887-697aabc4c7f8 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:09.645219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 686/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:11.693965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f023811-058f-4f8a-8000-dc98a2719b3e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:11.694683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f023811-058f-4f8a-8000-dc98a2719b3e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:11.694801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f023811-058f-4f8a-8000-dc98a2719b3e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:11.696449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 694/1380] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:11.701660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17181884-e804-4680-ab38-16b63a8ef8b7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:11.702126 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17181884-e804-4680-ab38-16b63a8ef8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:11.702217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17181884-e804-4680-ab38-16b63a8ef8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:11.713105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:11.777764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-17181884-e804-4680-ab38-16b63a8ef8b7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:11.778937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-17181884-e804-4680-ab38-16b63a8ef8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:11.911930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 687/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:11.958340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f023811-058f-4f8a-8000-dc98a2719b3e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:11.959988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9f023811-058f-4f8a-8000-dc98a2719b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:12.120195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:12.134011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f023811-058f-4f8a-8000-dc98a2719b3e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:12.252514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029aa969-f00d-4487-9d9a-0603f0b5f85d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:12.253226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aa969-f00d-4487-9d9a-0603f0b5f85d service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:12.253595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aa969-f00d-4487-9d9a-0603f0b5f85d service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:12.253979 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aa969-f00d-4487-9d9a-0603f0b5f85d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:12.255671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-029aa969-f00d-4487-9d9a-0603f0b5f85d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:12.349800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029aa969-f00d-4487-9d9a-0603f0b5f85d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:12.351816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029aa969-f00d-4487-9d9a-0603f0b5f85d service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:12.362813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:12.372562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 695/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:14.731460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b537b-7e81-403a-8b07-e31efa44ee8e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:14.732017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b537b-7e81-403a-8b07-e31efa44ee8e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:14.732017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b537b-7e81-403a-8b07-e31efa44ee8e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:14.733573 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 688/1383] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:14.740962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e610c-41a4-4284-be49-407927a0377d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:14.741445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e610c-41a4-4284-be49-407927a0377d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:14.741734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e610c-41a4-4284-be49-407927a0377d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:14.826961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-468e610c-41a4-4284-be49-407927a0377d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:14.828432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-468e610c-41a4-4284-be49-407927a0377d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:14.953145 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 696/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:50:15.002208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b537b-7e81-403a-8b07-e31efa44ee8e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:15.004207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-111b537b-7e81-403a-8b07-e31efa44ee8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:15.118495 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:15.193018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b537b-7e81-403a-8b07-e31efa44ee8e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:15.316368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05429a44-ef6f-4890-aa42-86e1d791ba8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:15.316996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05429a44-ef6f-4890-aa42-86e1d791ba8d service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:15.317272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05429a44-ef6f-4890-aa42-86e1d791ba8d service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:15.317541 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05429a44-ef6f-4890-aa42-86e1d791ba8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:15.319271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-05429a44-ef6f-4890-aa42-86e1d791ba8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:15.373881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:15.420827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05429a44-ef6f-4890-aa42-86e1d791ba8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:15.422696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05429a44-ef6f-4890-aa42-86e1d791ba8d service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:15.445116 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 689/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:17.612613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af164ac9-493a-4e3a-bb32-092a7b6bed32 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:17.613161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af164ac9-493a-4e3a-bb32-092a7b6bed32 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:17.613161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af164ac9-493a-4e3a-bb32-092a7b6bed32 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:17.614576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 697/1386] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:17.620646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2f121-d993-4a9c-ac15-2f27a8d02073 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:17.621204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2f121-d993-4a9c-ac15-2f27a8d02073 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:17.621204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2f121-d993-4a9c-ac15-2f27a8d02073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:17.692077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c0e2f121-d993-4a9c-ac15-2f27a8d02073 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:17.693383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c0e2f121-d993-4a9c-ac15-2f27a8d02073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:17.795129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 690/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:17.846123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af164ac9-493a-4e3a-bb32-092a7b6bed32 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:17.848060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-af164ac9-493a-4e3a-bb32-092a7b6bed32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:17.928262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:18.017102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af164ac9-493a-4e3a-bb32-092a7b6bed32 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:18.152219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd364a02-b5fc-41c5-941c-ad7023517f17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:18.153412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd364a02-b5fc-41c5-941c-ad7023517f17 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:18.153772 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd364a02-b5fc-41c5-941c-ad7023517f17 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:18.154112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd364a02-b5fc-41c5-941c-ad7023517f17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:18.156374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bd364a02-b5fc-41c5-941c-ad7023517f17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:18.206336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:18.273305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd364a02-b5fc-41c5-941c-ad7023517f17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:18.275058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd364a02-b5fc-41c5-941c-ad7023517f17 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:18.299020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 698/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:20.423809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78e352-7cf4-4576-869b-bded8eddf522 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:20.424413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78e352-7cf4-4576-869b-bded8eddf522 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:20.424413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78e352-7cf4-4576-869b-bded8eddf522 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:20.425794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 691/1389] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:50:20.431210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d7721-762a-4240-a2f5-243f7ee4b153 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:20.431644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d7721-762a-4240-a2f5-243f7ee4b153 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:20.431734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d7721-762a-4240-a2f5-243f7ee4b153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:20.502531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-580d7721-762a-4240-a2f5-243f7ee4b153 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:20.503750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-580d7721-762a-4240-a2f5-243f7ee4b153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:20.608659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 699/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:20.652913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78e352-7cf4-4576-869b-bded8eddf522 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:20.654524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ec78e352-7cf4-4576-869b-bded8eddf522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:20.689816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:20.810947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78e352-7cf4-4576-869b-bded8eddf522 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:20.918433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:20.931089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12684c02-65b8-4b5c-ac86-75fdb9a29010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:20.931777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12684c02-65b8-4b5c-ac86-75fdb9a29010 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:20.932054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12684c02-65b8-4b5c-ac86-75fdb9a29010 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:20.932360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12684c02-65b8-4b5c-ac86-75fdb9a29010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:20.934011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-12684c02-65b8-4b5c-ac86-75fdb9a29010 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:21.025421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12684c02-65b8-4b5c-ac86-75fdb9a29010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:21.026945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12684c02-65b8-4b5c-ac86-75fdb9a29010 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:21.046202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 692/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:21.099536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dae0d9-4468-474a-b00b-2c7863110338 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:21.099784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dae0d9-4468-474a-b00b-2c7863110338 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:21.100149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dae0d9-4468-474a-b00b-2c7863110338 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:21.101943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 700/1392] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:21.107509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005053b-6332-4e5a-aa54-99d63ddf4963 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:21.107804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005053b-6332-4e5a-aa54-99d63ddf4963 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:21.108084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005053b-6332-4e5a-aa54-99d63ddf4963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:21.183518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8005053b-6332-4e5a-aa54-99d63ddf4963 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:21.184915 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8005053b-6332-4e5a-aa54-99d63ddf4963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:21.252786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:21.291472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 693/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:21.346354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f93e-3a61-40e4-97d0-b730a4422dcd None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:21.346639 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f93e-3a61-40e4-97d0-b730a4422dcd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:21.346908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f93e-3a61-40e4-97d0-b730a4422dcd None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:21.348447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 701/1394] 10.222.0.20 () {54 vars in 962 bytes} [Mon Jul 11 20:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:21.353858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7f11-6472-4603-943b-ef5f7ef79bac None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:21.354145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7f11-6472-4603-943b-ef5f7ef79bac None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:21.354431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7f11-6472-4603-943b-ef5f7ef79bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:21.433591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c1be7f11-6472-4603-943b-ef5f7ef79bac None None] MFA Rules not processed for user `097b12fbf6494336b11ece5c498a4103`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:21.434813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c1be7f11-6472-4603-943b-ef5f7ef79bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:21.513511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 694/1395] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Jul 11 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:21.521207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522f93e-3a61-40e4-97d0-b730a4422dcd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:21.526849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0522f93e-3a61-40e4-97d0-b730a4422dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:21.545490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:21.641075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522f93e-3a61-40e4-97d0-b730a4422dcd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:21.720790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe910f-5988-4487-a074-3ee3e4d8a81c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097b12fbf6494336b11ece5c498a4103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:21.721436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe910f-5988-4487-a074-3ee3e4d8a81c service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:21.721711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe910f-5988-4487-a074-3ee3e4d8a81c service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:21.721977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe910f-5988-4487-a074-3ee3e4d8a81c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:21.723595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1dfe910f-5988-4487-a074-3ee3e4d8a81c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:21.793230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:21.837739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe910f-5988-4487-a074-3ee3e4d8a81c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:21.839234 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe910f-5988-4487-a074-3ee3e4d8a81c service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:21.858239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 702/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:22.575440 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb37417-2971-4755-9714-6b518457b1b0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:22.575947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb37417-2971-4755-9714-6b518457b1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:22.576045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb37417-2971-4755-9714-6b518457b1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:22.642226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7eb37417-2971-4755-9714-6b518457b1b0 None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:22.643426 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7eb37417-2971-4755-9714-6b518457b1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:22.652030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:22.734333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 695/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:22.780347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548bbc6-470a-4442-b08e-b4467e0e7380 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:22.780829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548bbc6-470a-4442-b08e-b4467e0e7380 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:22.781026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548bbc6-470a-4442-b08e-b4467e0e7380 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:22.782548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 703/1398] 10.222.0.20 () {54 vars in 942 bytes} [Mon Jul 11 20:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:22.788878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d004cb5-87a5-4ce0-965d-aeb5724f72ff None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:22.789188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d004cb5-87a5-4ce0-965d-aeb5724f72ff None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:22.789452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d004cb5-87a5-4ce0-965d-aeb5724f72ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:22.872925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1d004cb5-87a5-4ce0-965d-aeb5724f72ff None None] MFA Rules not processed for user `97b5f28d84fe4570a7c5c904dc4e502c`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:22.874148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1d004cb5-87a5-4ce0-965d-aeb5724f72ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:22.966383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:22.990279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 696/1399] 10.222.0.20 () {58 vars in 1039 bytes} [Mon Jul 11 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:50:22.999944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548bbc6-470a-4442-b08e-b4467e0e7380 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:23.001612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a548bbc6-470a-4442-b08e-b4467e0e7380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:23.114449 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390fb47-df7a-45ab-b3c0-7c9504074af1 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:23.114773 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390fb47-df7a-45ab-b3c0-7c9504074af1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:23.115052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390fb47-df7a-45ab-b3c0-7c9504074af1 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:23.116690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 697/1400] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 20:50:23.121582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2122b82-c795-4e03-add6-a15f310656c6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:23.121868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2122b82-c795-4e03-add6-a15f310656c6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:23.122116 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2122b82-c795-4e03-add6-a15f310656c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:23.151935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548bbc6-470a-4442-b08e-b4467e0e7380 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:23.189089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f2122b82-c795-4e03-add6-a15f310656c6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:23.189742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:23.190684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f2122b82-c795-4e03-add6-a15f310656c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:23.264959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea8c00-8ff9-4f13-aaca-6a2974c63601 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:23.265732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea8c00-8ff9-4f13-aaca-6a2974c63601 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:23.266082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea8c00-8ff9-4f13-aaca-6a2974c63601 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:23.266531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea8c00-8ff9-4f13-aaca-6a2974c63601 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:23.268134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-25ea8c00-8ff9-4f13-aaca-6a2974c63601 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:23.295100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 698/1401] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:50:23.340296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2122b82-c795-4e03-add6-a15f310656c6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:23.341577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f2122b82-c795-4e03-add6-a15f310656c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:23.352024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea8c00-8ff9-4f13-aaca-6a2974c63601 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:23.353695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea8c00-8ff9-4f13-aaca-6a2974c63601 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:23.361983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 704/1402] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 20:50:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 20:50:23.413849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:23.470822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2122b82-c795-4e03-add6-a15f310656c6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:23.602510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8489c8-24fa-4f11-a24c-68f0f3cccc12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:23.603167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8489c8-24fa-4f11-a24c-68f0f3cccc12 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:23.603350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8489c8-24fa-4f11-a24c-68f0f3cccc12 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:23.603612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8489c8-24fa-4f11-a24c-68f0f3cccc12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:23.605330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cb8489c8-24fa-4f11-a24c-68f0f3cccc12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:23.664980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:23.701772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8489c8-24fa-4f11-a24c-68f0f3cccc12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:23.703387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8489c8-24fa-4f11-a24c-68f0f3cccc12 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:23.721765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 699/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:25.852336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2308c-5539-466f-b6e7-810ad0cddf7b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:25.853326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2308c-5539-466f-b6e7-810ad0cddf7b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:25.853605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2308c-5539-466f-b6e7-810ad0cddf7b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:25.855010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 705/1404] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:25.860613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea767fb0-ed75-4f9d-80b0-07bb8b8c111b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:25.861076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea767fb0-ed75-4f9d-80b0-07bb8b8c111b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:25.861169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea767fb0-ed75-4f9d-80b0-07bb8b8c111b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:25.931506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ea767fb0-ed75-4f9d-80b0-07bb8b8c111b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:25.932806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ea767fb0-ed75-4f9d-80b0-07bb8b8c111b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:26.036578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 700/1405] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:26.083171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2308c-5539-466f-b6e7-810ad0cddf7b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:26.084820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dac2308c-5539-466f-b6e7-810ad0cddf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:26.160018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:26.246548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2308c-5539-466f-b6e7-810ad0cddf7b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:26.370985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2014-dd9f-45d7-931e-6a9bc422a3c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:26.371764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2014-dd9f-45d7-931e-6a9bc422a3c7 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:26.372041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2014-dd9f-45d7-931e-6a9bc422a3c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:26.372328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2014-dd9f-45d7-931e-6a9bc422a3c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:26.374165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-596d2014-dd9f-45d7-931e-6a9bc422a3c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:26.405027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:26.474251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d2014-dd9f-45d7-931e-6a9bc422a3c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:26.476313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d2014-dd9f-45d7-931e-6a9bc422a3c7 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:26.501593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 706/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:28.603521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bf98e-1609-43ec-809a-792560049618 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:28.604091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bf98e-1609-43ec-809a-792560049618 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:28.604091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bf98e-1609-43ec-809a-792560049618 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:28.605670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 701/1407] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:28.613838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10630f4e-16ed-4877-95f0-cd8c20c1628e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:28.613838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10630f4e-16ed-4877-95f0-cd8c20c1628e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:28.613838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10630f4e-16ed-4877-95f0-cd8c20c1628e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:28.684455 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-10630f4e-16ed-4877-95f0-cd8c20c1628e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:28.686258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-10630f4e-16ed-4877-95f0-cd8c20c1628e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:28.796250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 707/1408] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:28.838907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bf98e-1609-43ec-809a-792560049618 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:28.840493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-380bf98e-1609-43ec-809a-792560049618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:28.880741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:29.008450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bf98e-1609-43ec-809a-792560049618 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:29.129082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:29.138709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57c692-6f72-4159-a297-cea1e3a7b384 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:29.139388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57c692-6f72-4159-a297-cea1e3a7b384 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:29.139671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57c692-6f72-4159-a297-cea1e3a7b384 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:29.139948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57c692-6f72-4159-a297-cea1e3a7b384 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:29.141730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ea57c692-6f72-4159-a297-cea1e3a7b384 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:29.242867 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57c692-6f72-4159-a297-cea1e3a7b384 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:29.244514 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57c692-6f72-4159-a297-cea1e3a7b384 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:29.264996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 702/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:38.793469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:39.262004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=47 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:45.329866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15444b95-5cd1-4863-8e67-2099e5acb9d5 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:45.330769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15444b95-5cd1-4863-8e67-2099e5acb9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:45.330769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15444b95-5cd1-4863-8e67-2099e5acb9d5 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:45.332696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 708/1410] 10.222.0.20 () {54 vars in 918 bytes} [Mon Jul 11 20:50:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:45.344929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad99f9a-4955-42e9-99f6-bcb4b8ae990e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:45.345747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad99f9a-4955-42e9-99f6-bcb4b8ae990e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:45.345914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad99f9a-4955-42e9-99f6-bcb4b8ae990e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:45.427611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4ad99f9a-4955-42e9-99f6-bcb4b8ae990e None None] MFA Rules not processed for user `946d11b552754364b55eb495f68ab852`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:45.428881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4ad99f9a-4955-42e9-99f6-bcb4b8ae990e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:45.547250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 703/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Jul 11 20:50:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:50:45.562597 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f7fca-1517-40d8-ac5d-837d5d9c67cd None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:45.562879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f7fca-1517-40d8-ac5d-837d5d9c67cd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:45.563150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f7fca-1517-40d8-ac5d-837d5d9c67cd None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:45.564568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 709/1412] 10.222.0.20 () {54 vars in 918 bytes} [Mon Jul 11 20:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:45.569732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81795566-31e5-4550-8b60-c24f9a33c440 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:45.570133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81795566-31e5-4550-8b60-c24f9a33c440 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:45.570357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81795566-31e5-4550-8b60-c24f9a33c440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:45.654376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:45.659752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-81795566-31e5-4550-8b60-c24f9a33c440 None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:45.661495 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-81795566-31e5-4550-8b60-c24f9a33c440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:45.813800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 704/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Jul 11 20:50:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:48.290179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb0971-2a03-4205-ba52-cc44e75c1c80 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:48.291100 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb0971-2a03-4205-ba52-cc44e75c1c80 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:48.291100 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb0971-2a03-4205-ba52-cc44e75c1c80 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:48.293312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:50:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:48.298819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8a548-a1be-4fda-b48a-8989cadd37f6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:48.299246 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8a548-a1be-4fda-b48a-8989cadd37f6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:48.299361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8a548-a1be-4fda-b48a-8989cadd37f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:48.373052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ded8a548-a1be-4fda-b48a-8989cadd37f6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:48.374262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ded8a548-a1be-4fda-b48a-8989cadd37f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:48.430192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:48.476200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:48.522377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb0971-2a03-4205-ba52-cc44e75c1c80 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:48.524063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-51bb0971-2a03-4205-ba52-cc44e75c1c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:48.687873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb0971-2a03-4205-ba52-cc44e75c1c80 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:48.769608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:48.816938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61faf8a-fe8f-4141-a7c4-f907b7d8afcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:48.817582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61faf8a-fe8f-4141-a7c4-f907b7d8afcd service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:48.817879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61faf8a-fe8f-4141-a7c4-f907b7d8afcd service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:48.818150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61faf8a-fe8f-4141-a7c4-f907b7d8afcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:48.819791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e61faf8a-fe8f-4141-a7c4-f907b7d8afcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:48.913282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61faf8a-fe8f-4141-a7c4-f907b7d8afcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:48.915473 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61faf8a-fe8f-4141-a7c4-f907b7d8afcd service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:48.936867 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 711/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:49.839598 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3b5a9-b986-4373-af5f-6095271a1c9c None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:49.840148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3b5a9-b986-4373-af5f-6095271a1c9c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:49.840148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3b5a9-b986-4373-af5f-6095271a1c9c None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:49.841657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:50:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:49.847014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfc39c-d9ce-4f6f-92cd-e829abf6de11 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:49.847583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfc39c-d9ce-4f6f-92cd-e829abf6de11 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:49.847583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfc39c-d9ce-4f6f-92cd-e829abf6de11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:49.926015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2adfc39c-d9ce-4f6f-92cd-e829abf6de11 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:49.927239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2adfc39c-d9ce-4f6f-92cd-e829abf6de11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:49.954898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:50.029411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:50.188049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3b5a9-b986-4373-af5f-6095271a1c9c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:50.190481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-35a3b5a9-b986-4373-af5f-6095271a1c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:50.364134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3b5a9-b986-4373-af5f-6095271a1c9c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:50.406575 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:50.489730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2734c-78f6-497e-a572-8944ce7df0b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:50.490299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2734c-78f6-497e-a572-8944ce7df0b1 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:50.490574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2734c-78f6-497e-a572-8944ce7df0b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:50.490848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2734c-78f6-497e-a572-8944ce7df0b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:50.492528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1df2734c-78f6-497e-a572-8944ce7df0b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:50.595891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2734c-78f6-497e-a572-8944ce7df0b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:50.597571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2734c-78f6-497e-a572-8944ce7df0b1 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:50.616968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 707/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:51.503123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678e20-9ba1-4e87-adb0-b0e450ab4a5d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:51.503724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678e20-9ba1-4e87-adb0-b0e450ab4a5d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:51.503724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678e20-9ba1-4e87-adb0-b0e450ab4a5d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:51.505829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:50:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:51.511313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187618f5-95a3-43e3-9bee-0301942f1917 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:51.511762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187618f5-95a3-43e3-9bee-0301942f1917 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:51.511882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187618f5-95a3-43e3-9bee-0301942f1917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:51.577334 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:51.604170 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-187618f5-95a3-43e3-9bee-0301942f1917 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:51.605421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-187618f5-95a3-43e3-9bee-0301942f1917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:51.714577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:51.874623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678e20-9ba1-4e87-adb0-b0e450ab4a5d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:51.877287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-70678e20-9ba1-4e87-adb0-b0e450ab4a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:52.040783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678e20-9ba1-4e87-adb0-b0e450ab4a5d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:52.053741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:52.162341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f0ed-623a-44df-91b2-83e276ae0f16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:52.162993 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f0ed-623a-44df-91b2-83e276ae0f16 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:52.163283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f0ed-623a-44df-91b2-83e276ae0f16 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:52.163549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f0ed-623a-44df-91b2-83e276ae0f16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:52.165257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-66e2f0ed-623a-44df-91b2-83e276ae0f16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:52.264796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f0ed-623a-44df-91b2-83e276ae0f16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:52.266416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f0ed-623a-44df-91b2-83e276ae0f16 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:52.284935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 714/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:53.149123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2cbed-ea06-4112-9c04-2e5e1ebb2c3b None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:53.149677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2cbed-ea06-4112-9c04-2e5e1ebb2c3b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:53.149677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2cbed-ea06-4112-9c04-2e5e1ebb2c3b None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:53.151212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:53.157363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95322b4b-b543-4a08-a370-932b1f4fe237 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:53.157842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95322b4b-b543-4a08-a370-932b1f4fe237 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:53.157972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95322b4b-b543-4a08-a370-932b1f4fe237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:53.180052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:53.239852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-95322b4b-b543-4a08-a370-932b1f4fe237 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:53.241094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-95322b4b-b543-4a08-a370-932b1f4fe237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:53.343647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:53.495464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2cbed-ea06-4112-9c04-2e5e1ebb2c3b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:53.497868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-04f2cbed-ea06-4112-9c04-2e5e1ebb2c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:53.615174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:53.654296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2cbed-ea06-4112-9c04-2e5e1ebb2c3b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:53.773821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155fcd9-ee9c-493c-8cbf-6dc6aea84980 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:53.774395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155fcd9-ee9c-493c-8cbf-6dc6aea84980 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:53.774680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155fcd9-ee9c-493c-8cbf-6dc6aea84980 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:53.774953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155fcd9-ee9c-493c-8cbf-6dc6aea84980 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:53.776628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d155fcd9-ee9c-493c-8cbf-6dc6aea84980 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:53.861997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:53.876741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155fcd9-ee9c-493c-8cbf-6dc6aea84980 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:53.878436 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155fcd9-ee9c-493c-8cbf-6dc6aea84980 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:53.902829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 710/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:54.755825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515fb81-5482-4bc3-8587-9ef93ffaa08c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:54.756446 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515fb81-5482-4bc3-8587-9ef93ffaa08c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:54.756446 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515fb81-5482-4bc3-8587-9ef93ffaa08c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:54.757689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:54.763908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd99b46-a544-45c2-bd17-24d98c7f9df2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:54.764636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd99b46-a544-45c2-bd17-24d98c7f9df2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:54.764875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd99b46-a544-45c2-bd17-24d98c7f9df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:54.852791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9bd99b46-a544-45c2-bd17-24d98c7f9df2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:54.854020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9bd99b46-a544-45c2-bd17-24d98c7f9df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:54.961647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:55.114413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515fb81-5482-4bc3-8587-9ef93ffaa08c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:55.116888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2515fb81-5482-4bc3-8587-9ef93ffaa08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:55.216529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:55.283839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515fb81-5482-4bc3-8587-9ef93ffaa08c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:55.403605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb015207-2bd5-4a54-b9cd-4f4f3f4cdc57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:55.404137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb015207-2bd5-4a54-b9cd-4f4f3f4cdc57 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:55.404460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb015207-2bd5-4a54-b9cd-4f4f3f4cdc57 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:55.404734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb015207-2bd5-4a54-b9cd-4f4f3f4cdc57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:55.406363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cb015207-2bd5-4a54-b9cd-4f4f3f4cdc57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:55.456619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:55.502863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb015207-2bd5-4a54-b9cd-4f4f3f4cdc57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:55.504551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb015207-2bd5-4a54-b9cd-4f4f3f4cdc57 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:55.522930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 717/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:56.334243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9106-2346-426d-a6d7-1ba7c88e6028 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:56.334842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9106-2346-426d-a6d7-1ba7c88e6028 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:56.334842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9106-2346-426d-a6d7-1ba7c88e6028 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:56.336339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:56.345298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e1c1-eda8-406c-a809-9830470d8a39 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:56.345889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e1c1-eda8-406c-a809-9830470d8a39 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:56.346534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e1c1-eda8-406c-a809-9830470d8a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:56.427394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fd81e1c1-eda8-406c-a809-9830470d8a39 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:56.428623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fd81e1c1-eda8-406c-a809-9830470d8a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:56.539871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:56.694701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf9106-2346-426d-a6d7-1ba7c88e6028 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:56.697526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-46cf9106-2346-426d-a6d7-1ba7c88e6028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:56.750409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:56.854970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf9106-2346-426d-a6d7-1ba7c88e6028 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:56.982526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44332fd4-9a3f-49f1-b25c-46f292a78400 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:56.983089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332fd4-9a3f-49f1-b25c-46f292a78400 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:56.983368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332fd4-9a3f-49f1-b25c-46f292a78400 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:56.983640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332fd4-9a3f-49f1-b25c-46f292a78400 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:56.985339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-44332fd4-9a3f-49f1-b25c-46f292a78400 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:56.992337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:57.080154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44332fd4-9a3f-49f1-b25c-46f292a78400 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:57.081795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44332fd4-9a3f-49f1-b25c-46f292a78400 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:57.100360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 713/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:57.898843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd3d8e-702e-4224-9030-8230a8d2c659 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:57.899744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd3d8e-702e-4224-9030-8230a8d2c659 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:57.899915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd3d8e-702e-4224-9030-8230a8d2c659 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:57.902168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:50:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:57.910250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac19bd1-e4be-4319-92dc-f19ea2495faa None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:57.910697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac19bd1-e4be-4319-92dc-f19ea2495faa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:57.911067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac19bd1-e4be-4319-92dc-f19ea2495faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:57.995954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9ac19bd1-e4be-4319-92dc-f19ea2495faa None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:57.997204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9ac19bd1-e4be-4319-92dc-f19ea2495faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:58.075480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:58.106553 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:50:58.255095 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd3d8e-702e-4224-9030-8230a8d2c659 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:58.257551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-98fd3d8e-702e-4224-9030-8230a8d2c659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:58.282317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:58.422396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd3d8e-702e-4224-9030-8230a8d2c659 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:50:58.526175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:58.554948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d7b83-df10-49f5-b4fa-9597084164a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:50:58.555797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d7b83-df10-49f5-b4fa-9597084164a0 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:58.556084 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d7b83-df10-49f5-b4fa-9597084164a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:58.556390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d7b83-df10-49f5-b4fa-9597084164a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:58.558037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-951d7b83-df10-49f5-b4fa-9597084164a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:58.656721 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d7b83-df10-49f5-b4fa-9597084164a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:50:58.658350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d7b83-df10-49f5-b4fa-9597084164a0 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:50:58.676669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 720/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:50:59.488032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5e6d5-87ae-444c-ab48-1fbd1cc7472e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:59.488971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5e6d5-87ae-444c-ab48-1fbd1cc7472e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:59.488971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5e6d5-87ae-444c-ab48-1fbd1cc7472e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:59.491109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:50:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:50:59.498429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec196726-3eb3-476d-8e03-cbdc031354ed None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:50:59.498867 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec196726-3eb3-476d-8e03-cbdc031354ed None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:50:59.498974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec196726-3eb3-476d-8e03-cbdc031354ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:50:59.587386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ec196726-3eb3-476d-8e03-cbdc031354ed None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:50:59.588930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ec196726-3eb3-476d-8e03-cbdc031354ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:50:59.632734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:50:59.694108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:50:59.846170 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5e6d5-87ae-444c-ab48-1fbd1cc7472e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:50:59.847765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a2c5e6d5-87ae-444c-ab48-1fbd1cc7472e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:00.001679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5e6d5-87ae-444c-ab48-1fbd1cc7472e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:00.058017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:00.119567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf4bdb-6e83-463e-a45b-b256b7d3481c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:00.120253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf4bdb-6e83-463e-a45b-b256b7d3481c service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:00.120536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf4bdb-6e83-463e-a45b-b256b7d3481c service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:00.120812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf4bdb-6e83-463e-a45b-b256b7d3481c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:00.122464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fabf4bdb-6e83-463e-a45b-b256b7d3481c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:00.217529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf4bdb-6e83-463e-a45b-b256b7d3481c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:00.219156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf4bdb-6e83-463e-a45b-b256b7d3481c service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:00.240291 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 716/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:50:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:01.109189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814293a1-4320-4331-a151-e6a83008c0f1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:01.109766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814293a1-4320-4331-a151-e6a83008c0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:01.109766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814293a1-4320-4331-a151-e6a83008c0f1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:01.111136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:01.117321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fbad9-f9f2-4158-873d-7b4aea6bb1b4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:01.118311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fbad9-f9f2-4158-873d-7b4aea6bb1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:01.118311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fbad9-f9f2-4158-873d-7b4aea6bb1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:01.203520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-531fbad9-f9f2-4158-873d-7b4aea6bb1b4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:01.205292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-531fbad9-f9f2-4158-873d-7b4aea6bb1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:01.208063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:01.306053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:01.474299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814293a1-4320-4331-a151-e6a83008c0f1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:01.476688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-814293a1-4320-4331-a151-e6a83008c0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:01.647278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814293a1-4320-4331-a151-e6a83008c0f1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:01.675697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:01.767229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5581f3-1035-414e-bfb8-406f6df3b56b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:01.767820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5581f3-1035-414e-bfb8-406f6df3b56b service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:01.768101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5581f3-1035-414e-bfb8-406f6df3b56b service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:01.768420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5581f3-1035-414e-bfb8-406f6df3b56b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:01.770098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5a5581f3-1035-414e-bfb8-406f6df3b56b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:01.867126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5581f3-1035-414e-bfb8-406f6df3b56b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:01.868929 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5581f3-1035-414e-bfb8-406f6df3b56b service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:01.888594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 723/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:03.851706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6b81b-a774-4f69-83c1-380d6aec2f1a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:03.852221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6b81b-a774-4f69-83c1-380d6aec2f1a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:03.852320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6b81b-a774-4f69-83c1-380d6aec2f1a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:03.853865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:03.859419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf1abd-9419-4291-a7c1-675eb3f84191 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:03.859910 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf1abd-9419-4291-a7c1-675eb3f84191 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:03.860024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf1abd-9419-4291-a7c1-675eb3f84191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:03.906160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:03.935512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d6cf1abd-9419-4291-a7c1-675eb3f84191 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:03.936767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d6cf1abd-9419-4291-a7c1-675eb3f84191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:04.048676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:04.098737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6b81b-a774-4f69-83c1-380d6aec2f1a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:04.100334 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e5c6b81b-a774-4f69-83c1-380d6aec2f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:04.212850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6b81b-a774-4f69-83c1-380d6aec2f1a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:04.257328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:04.311527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035db38f-7dfe-4aad-a84c-dbbec801e839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:04.312223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035db38f-7dfe-4aad-a84c-dbbec801e839 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:04.312499 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035db38f-7dfe-4aad-a84c-dbbec801e839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:04.312772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035db38f-7dfe-4aad-a84c-dbbec801e839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:04.314420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-035db38f-7dfe-4aad-a84c-dbbec801e839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:04.449285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035db38f-7dfe-4aad-a84c-dbbec801e839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:04.451843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035db38f-7dfe-4aad-a84c-dbbec801e839 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:04.471883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 719/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:06.707605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde79d5c-a3b3-4bdf-a68e-0f7203eea2b6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:06.708536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde79d5c-a3b3-4bdf-a68e-0f7203eea2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:06.708536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde79d5c-a3b3-4bdf-a68e-0f7203eea2b6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:06.710673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:51:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:06.717199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba43cb7-6195-4b82-bd0f-f7a7ce6a51da None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:06.717988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba43cb7-6195-4b82-bd0f-f7a7ce6a51da None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:06.718144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba43cb7-6195-4b82-bd0f-f7a7ce6a51da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:06.788567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bba43cb7-6195-4b82-bd0f-f7a7ce6a51da None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:06.789781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bba43cb7-6195-4b82-bd0f-f7a7ce6a51da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:06.803718 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:06.898428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:06.936818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde79d5c-a3b3-4bdf-a68e-0f7203eea2b6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:06.939174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cde79d5c-a3b3-4bdf-a68e-0f7203eea2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:07.052796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde79d5c-a3b3-4bdf-a68e-0f7203eea2b6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:07.066057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:07.133135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d1d42-ebf2-4d24-845b-40dea443fe97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:07.133789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d1d42-ebf2-4d24-845b-40dea443fe97 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:07.134107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d1d42-ebf2-4d24-845b-40dea443fe97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:07.134454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d1d42-ebf2-4d24-845b-40dea443fe97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:07.136157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-141d1d42-ebf2-4d24-845b-40dea443fe97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:07.253113 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d1d42-ebf2-4d24-845b-40dea443fe97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:07.254559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d1d42-ebf2-4d24-845b-40dea443fe97 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:07.273611 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 726/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:09.715458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5f24a-1696-4cbd-b751-a26465068c7c None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:09.716356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5f24a-1696-4cbd-b751-a26465068c7c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:09.716356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5f24a-1696-4cbd-b751-a26465068c7c None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:09.718728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:51:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:09.724096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2fa3-83d9-48e2-934e-bead3789ea71 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:09.724573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2fa3-83d9-48e2-934e-bead3789ea71 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:09.724676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2fa3-83d9-48e2-934e-bead3789ea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:09.774562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:09.794932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-927a2fa3-83d9-48e2-934e-bead3789ea71 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:09.796394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-927a2fa3-83d9-48e2-934e-bead3789ea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:09.907374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:09.949574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5f24a-1696-4cbd-b751-a26465068c7c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:09.951136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-23b5f24a-1696-4cbd-b751-a26465068c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:10.075007 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5f24a-1696-4cbd-b751-a26465068c7c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:10.117051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:10.153846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60084f2-659e-4feb-b2e5-be385d1ff9ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:10.154498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60084f2-659e-4feb-b2e5-be385d1ff9ef service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:10.154777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60084f2-659e-4feb-b2e5-be385d1ff9ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:10.155054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60084f2-659e-4feb-b2e5-be385d1ff9ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:10.156699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c60084f2-659e-4feb-b2e5-be385d1ff9ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:10.267928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60084f2-659e-4feb-b2e5-be385d1ff9ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:10.269401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60084f2-659e-4feb-b2e5-be385d1ff9ef service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:10.287681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 722/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:12.426886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdd0ad-105b-419c-aed0-50b186b5e262 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:12.427819 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdd0ad-105b-419c-aed0-50b186b5e262 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:12.427819 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdd0ad-105b-419c-aed0-50b186b5e262 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:12.430034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:51:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:12.435576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5904a-adbc-4def-aac4-c94861c0f732 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:12.435996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5904a-adbc-4def-aac4-c94861c0f732 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:12.436135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5904a-adbc-4def-aac4-c94861c0f732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:12.507239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bde5904a-adbc-4def-aac4-c94861c0f732 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:12.508497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bde5904a-adbc-4def-aac4-c94861c0f732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:12.530792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:12.609587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:12.648074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdd0ad-105b-419c-aed0-50b186b5e262 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:12.650569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-02bdd0ad-105b-419c-aed0-50b186b5e262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:12.774550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdd0ad-105b-419c-aed0-50b186b5e262 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:12.796157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:12.860254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e193e2-9ef1-45d2-bfd3-61ffc25c39c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:12.860908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e193e2-9ef1-45d2-bfd3-61ffc25c39c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:12.861192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e193e2-9ef1-45d2-bfd3-61ffc25c39c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:12.861464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e193e2-9ef1-45d2-bfd3-61ffc25c39c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:12.863112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-41e193e2-9ef1-45d2-bfd3-61ffc25c39c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:12.986115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e193e2-9ef1-45d2-bfd3-61ffc25c39c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:12.987587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e193e2-9ef1-45d2-bfd3-61ffc25c39c0 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:13.006330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 729/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:15.142614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ca07e-d77b-4627-abe5-9b06c2e2f4ef None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:15.143175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ca07e-d77b-4627-abe5-9b06c2e2f4ef None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:15.143551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ca07e-d77b-4627-abe5-9b06c2e2f4ef None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:15.146114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:51:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:15.152717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114012ae-7ffe-4411-8e6c-3accaec368ac None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:15.153158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114012ae-7ffe-4411-8e6c-3accaec368ac None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:15.153252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114012ae-7ffe-4411-8e6c-3accaec368ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:15.233649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:15.237018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-114012ae-7ffe-4411-8e6c-3accaec368ac None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:15.239240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-114012ae-7ffe-4411-8e6c-3accaec368ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:15.351860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:15.397434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ca07e-d77b-4627-abe5-9b06c2e2f4ef None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:15.399457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-043ca07e-d77b-4627-abe5-9b06c2e2f4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:15.522366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ca07e-d77b-4627-abe5-9b06c2e2f4ef None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:15.582261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:15.617604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fc01e-876d-40a7-be1e-ac1019fb0fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:15.618464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fc01e-876d-40a7-be1e-ac1019fb0fdd service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:15.618774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fc01e-876d-40a7-be1e-ac1019fb0fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:15.619110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fc01e-876d-40a7-be1e-ac1019fb0fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:15.621920 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fc9fc01e-876d-40a7-be1e-ac1019fb0fdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:15.736075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fc01e-876d-40a7-be1e-ac1019fb0fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:15.737476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fc01e-876d-40a7-be1e-ac1019fb0fdd service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:15.757445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 725/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:51:17.932332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce0c2-6223-4365-bbf6-4a63b2501364 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:17.932912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce0c2-6223-4365-bbf6-4a63b2501364 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:17.932912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce0c2-6223-4365-bbf6-4a63b2501364 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:17.935128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:51:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:17.944314 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4417d-391c-4b2a-ad7f-94f47af3c22e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:17.945087 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4417d-391c-4b2a-ad7f-94f47af3c22e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:17.945276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4417d-391c-4b2a-ad7f-94f47af3c22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:18.015138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-86d4417d-391c-4b2a-ad7f-94f47af3c22e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:18.016429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-86d4417d-391c-4b2a-ad7f-94f47af3c22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:18.050002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:18.125171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:18.166141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ce0c2-6223-4365-bbf6-4a63b2501364 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:18.167901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-683ce0c2-6223-4365-bbf6-4a63b2501364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:18.288705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ce0c2-6223-4365-bbf6-4a63b2501364 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:18.319249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:18.372976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104f54f-8407-4821-9c41-7d1f5f10727d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:18.373628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104f54f-8407-4821-9c41-7d1f5f10727d service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:18.373911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104f54f-8407-4821-9c41-7d1f5f10727d service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:18.374183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104f54f-8407-4821-9c41-7d1f5f10727d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:18.375907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b104f54f-8407-4821-9c41-7d1f5f10727d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:18.494944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104f54f-8407-4821-9c41-7d1f5f10727d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:18.496434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104f54f-8407-4821-9c41-7d1f5f10727d service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:18.514736 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 732/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:20.639464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41532b05-5047-4e6a-bb2c-c84ddaea1ce6 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:20.640042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41532b05-5047-4e6a-bb2c-c84ddaea1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:20.640042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41532b05-5047-4e6a-bb2c-c84ddaea1ce6 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:20.641449 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:20.647848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3e72b-5473-4f94-8dc7-d50b3b15c03f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:20.648598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3e72b-5473-4f94-8dc7-d50b3b15c03f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:20.648762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3e72b-5473-4f94-8dc7-d50b3b15c03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:20.718431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-50c3e72b-5473-4f94-8dc7-d50b3b15c03f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:20.719626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-50c3e72b-5473-4f94-8dc7-d50b3b15c03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:20.728304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:20.830934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:20.873509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41532b05-5047-4e6a-bb2c-c84ddaea1ce6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:20.875890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-41532b05-5047-4e6a-bb2c-c84ddaea1ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:20.985729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41532b05-5047-4e6a-bb2c-c84ddaea1ce6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:21.051277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:21.077419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf057496-8453-465f-a2b0-0223f112c033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:21.078078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf057496-8453-465f-a2b0-0223f112c033 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:21.078355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf057496-8453-465f-a2b0-0223f112c033 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:21.078620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf057496-8453-465f-a2b0-0223f112c033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:21.080310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cf057496-8453-465f-a2b0-0223f112c033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:21.195580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf057496-8453-465f-a2b0-0223f112c033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:21.197079 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf057496-8453-465f-a2b0-0223f112c033 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:21.216375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 728/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:23.314955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d257f-06f7-4938-9b7d-1d23e5b1c6e6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:23.315552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d257f-06f7-4938-9b7d-1d23e5b1c6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:23.315552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d257f-06f7-4938-9b7d-1d23e5b1c6e6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:23.316905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:23.322945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2c8e3-7b97-4a62-9aba-1ab1df6eda66 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:23.323430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2c8e3-7b97-4a62-9aba-1ab1df6eda66 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:23.323536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2c8e3-7b97-4a62-9aba-1ab1df6eda66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:23.389897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1bb2c8e3-7b97-4a62-9aba-1ab1df6eda66 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:23.391121 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1bb2c8e3-7b97-4a62-9aba-1ab1df6eda66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:23.431589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:23.495608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:23.543686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d257f-06f7-4938-9b7d-1d23e5b1c6e6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:23.545265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7d9d257f-06f7-4938-9b7d-1d23e5b1c6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:23.657054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d257f-06f7-4938-9b7d-1d23e5b1c6e6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:23.696915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:23.735471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fd43d-b00c-4438-98bc-824386757179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:23.736154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fd43d-b00c-4438-98bc-824386757179 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:23.736437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fd43d-b00c-4438-98bc-824386757179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:23.736716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fd43d-b00c-4438-98bc-824386757179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:23.738350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-491fd43d-b00c-4438-98bc-824386757179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:23.858399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fd43d-b00c-4438-98bc-824386757179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:23.859852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fd43d-b00c-4438-98bc-824386757179 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:23.879608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 735/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:25.886830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80bb57-6633-4170-8c34-2f55186f74de None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:25.887710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80bb57-6633-4170-8c34-2f55186f74de None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:25.887710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80bb57-6633-4170-8c34-2f55186f74de None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:25.890012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:51:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:25.898536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a847c-325c-4950-94bc-39d9a96d7f96 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:25.898952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a847c-325c-4950-94bc-39d9a96d7f96 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:25.899144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a847c-325c-4950-94bc-39d9a96d7f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:25.970929 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d62a847c-325c-4950-94bc-39d9a96d7f96 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:25.972206 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d62a847c-325c-4950-94bc-39d9a96d7f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:25.992530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:26.077709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:26.115130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80bb57-6633-4170-8c34-2f55186f74de None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:26.117619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cd80bb57-6633-4170-8c34-2f55186f74de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:26.244965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80bb57-6633-4170-8c34-2f55186f74de None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:26.329446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:26.339963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7521d-d87a-4a5f-8bcd-0257c1051185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:26.340866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7521d-d87a-4a5f-8bcd-0257c1051185 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:26.341226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7521d-d87a-4a5f-8bcd-0257c1051185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:26.341558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7521d-d87a-4a5f-8bcd-0257c1051185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:26.344066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a5f7521d-d87a-4a5f-8bcd-0257c1051185 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:26.466549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7521d-d87a-4a5f-8bcd-0257c1051185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:26.467919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7521d-d87a-4a5f-8bcd-0257c1051185 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:26.486507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 731/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:28.529173 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833990f2-1d57-42ef-b8df-4edde070ae59 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:28.529687 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833990f2-1d57-42ef-b8df-4edde070ae59 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:28.529687 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833990f2-1d57-42ef-b8df-4edde070ae59 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:28.531126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:28.537648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67f336-9191-4e66-8b31-8bbb55a0e542 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:28.538407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67f336-9191-4e66-8b31-8bbb55a0e542 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:28.538562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67f336-9191-4e66-8b31-8bbb55a0e542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:28.609279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0e67f336-9191-4e66-8b31-8bbb55a0e542 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:28.610475 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0e67f336-9191-4e66-8b31-8bbb55a0e542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:28.662082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:28.720274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:28.759694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833990f2-1d57-42ef-b8df-4edde070ae59 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:28.761302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-833990f2-1d57-42ef-b8df-4edde070ae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:28.868141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833990f2-1d57-42ef-b8df-4edde070ae59 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:28.917004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:28.945503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358729bd-6d33-460f-a097-ea21859c1636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:28.946169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358729bd-6d33-460f-a097-ea21859c1636 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:28.946451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358729bd-6d33-460f-a097-ea21859c1636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:28.946836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358729bd-6d33-460f-a097-ea21859c1636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:28.948448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-358729bd-6d33-460f-a097-ea21859c1636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:29.078220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358729bd-6d33-460f-a097-ea21859c1636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:29.079680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358729bd-6d33-460f-a097-ea21859c1636 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:29.099011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 738/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:51:38.718016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=22 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:39.096342 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=66 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:48.752502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fee97-03ea-48ab-8fda-11fa88f4304a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:48.753397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fee97-03ea-48ab-8fda-11fa88f4304a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:48.753397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fee97-03ea-48ab-8fda-11fa88f4304a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:48.755877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 733/1471] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:51:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:48.763797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d17e6-aaa7-469a-a60e-ab754f0ea248 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:48.764287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d17e6-aaa7-469a-a60e-ab754f0ea248 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:48.764380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d17e6-aaa7-469a-a60e-ab754f0ea248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:48.835848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0d0d17e6-aaa7-469a-a60e-ab754f0ea248 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:48.837123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0d0d17e6-aaa7-469a-a60e-ab754f0ea248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:48.943817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 739/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:49.029111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc771ca9-37f2-491e-a3c9-7104b466ff8a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:49.029444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc771ca9-37f2-491e-a3c9-7104b466ff8a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:49.029792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc771ca9-37f2-491e-a3c9-7104b466ff8a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:49.031846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 734/1473] 10.222.0.20 () {54 vars in 962 bytes} [Mon Jul 11 20:51:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:49.036912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe27576-8f0b-425a-a5c7-16436a81b0f9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:49.037200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe27576-8f0b-425a-a5c7-16436a81b0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:49.037457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe27576-8f0b-425a-a5c7-16436a81b0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:49.112765 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-afe27576-8f0b-425a-a5c7-16436a81b0f9 None None] MFA Rules not processed for user `097b12fbf6494336b11ece5c498a4103`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:49.113966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-afe27576-8f0b-425a-a5c7-16436a81b0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:49.136454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:49.202152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 740/1474] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Jul 11 20:51:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:49.210110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc771ca9-37f2-491e-a3c9-7104b466ff8a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:49.211691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dc771ca9-37f2-491e-a3c9-7104b466ff8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:49.324164 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc771ca9-37f2-491e-a3c9-7104b466ff8a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:49.402214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2c39e-7f42-4b45-9b45-971fb575a15a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097b12fbf6494336b11ece5c498a4103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:49.402786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2c39e-7f42-4b45-9b45-971fb575a15a service glance] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:49.403070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2c39e-7f42-4b45-9b45-971fb575a15a service glance] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:49.403341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2c39e-7f42-4b45-9b45-971fb575a15a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:49.405042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-48e2c39e-7f42-4b45-9b45-971fb575a15a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:49.470418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:49.538659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2c39e-7f42-4b45-9b45-971fb575a15a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:49.540136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2c39e-7f42-4b45-9b45-971fb575a15a service glance] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:49.559674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 735/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:51.663866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca701bae-c995-4b74-a40b-2c3ac86ddc89 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:51.664500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca701bae-c995-4b74-a40b-2c3ac86ddc89 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:51.664500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca701bae-c995-4b74-a40b-2c3ac86ddc89 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:51.666118 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 741/1476] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:51.671585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa835f9-d3a1-4dfa-b757-3f5369af36ff None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:51.672051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa835f9-d3a1-4dfa-b757-3f5369af36ff None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:51.672175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa835f9-d3a1-4dfa-b757-3f5369af36ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:51.740592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-efa835f9-d3a1-4dfa-b757-3f5369af36ff None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:51.741807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-efa835f9-d3a1-4dfa-b757-3f5369af36ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:51.858086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 736/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:51.882540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca701bae-c995-4b74-a40b-2c3ac86ddc89 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:51.885404 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ca701bae-c995-4b74-a40b-2c3ac86ddc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:52.031600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca701bae-c995-4b74-a40b-2c3ac86ddc89 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:52.081139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:52.133477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307476a2-2982-4b79-b69d-c35dbb0b7c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:52.134076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307476a2-2982-4b79-b69d-c35dbb0b7c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:52.134354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307476a2-2982-4b79-b69d-c35dbb0b7c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:52.134623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307476a2-2982-4b79-b69d-c35dbb0b7c50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:52.135488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307476a2-2982-4b79-b69d-c35dbb0b7c50 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:52.137287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-307476a2-2982-4b79-b69d-c35dbb0b7c50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:51:52.137287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 11 20:51:52.137287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: Jul 11 20:51:52.138445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307476a2-2982-4b79-b69d-c35dbb0b7c50 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:52.158446 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 742/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jul 11 20:51:51 2022] GET /identity/v3/projects => generated 2553 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:51:54.254696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752a330-453f-4e3f-a21a-ceff7befdb69 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:54.255297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752a330-453f-4e3f-a21a-ceff7befdb69 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:54.255297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752a330-453f-4e3f-a21a-ceff7befdb69 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:54.256689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 737/1479] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:54.262587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3580472-dc6e-46cd-a073-8d3067de739e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:54.263048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3580472-dc6e-46cd-a073-8d3067de739e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:54.263147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3580472-dc6e-46cd-a073-8d3067de739e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:54.335414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d3580472-dc6e-46cd-a073-8d3067de739e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:54.336671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d3580472-dc6e-46cd-a073-8d3067de739e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:54.447284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 743/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:54.490719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752a330-453f-4e3f-a21a-ceff7befdb69 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:54.492276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1752a330-453f-4e3f-a21a-ceff7befdb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:54.502293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:54.643066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752a330-453f-4e3f-a21a-ceff7befdb69 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:54.737290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:54.771076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05137431-05c9-4606-8e79-7c7448daff25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:54.771725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05137431-05c9-4606-8e79-7c7448daff25 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:54.771999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05137431-05c9-4606-8e79-7c7448daff25 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:54.772280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05137431-05c9-4606-8e79-7c7448daff25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:54.773943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-05137431-05c9-4606-8e79-7c7448daff25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:54.867901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05137431-05c9-4606-8e79-7c7448daff25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:54.869881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05137431-05c9-4606-8e79-7c7448daff25 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:54.888377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 738/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:57.052716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b811e8-b8e9-4f70-9d5a-8c662db9edfe None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:57.053316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b811e8-b8e9-4f70-9d5a-8c662db9edfe None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:57.053316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b811e8-b8e9-4f70-9d5a-8c662db9edfe None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:57.054652 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 744/1482] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:57.061134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7209f-d39a-488d-8cda-f31fd5525abd None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:57.061882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7209f-d39a-488d-8cda-f31fd5525abd None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:57.062045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7209f-d39a-488d-8cda-f31fd5525abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:57.132214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7ea7209f-d39a-488d-8cda-f31fd5525abd None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:57.133413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7ea7209f-d39a-488d-8cda-f31fd5525abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:57.181765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:57.243380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 739/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:51:57.283912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b811e8-b8e9-4f70-9d5a-8c662db9edfe None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:57.286325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-83b811e8-b8e9-4f70-9d5a-8c662db9edfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:57.318798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:57.400644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b811e8-b8e9-4f70-9d5a-8c662db9edfe None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:51:57.475718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d01d74-e416-4bb9-b9f4-49c01e9d5da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:51:57.476391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d01d74-e416-4bb9-b9f4-49c01e9d5da7 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:57.476671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d01d74-e416-4bb9-b9f4-49c01e9d5da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:57.476936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d01d74-e416-4bb9-b9f4-49c01e9d5da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:57.478622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-69d01d74-e416-4bb9-b9f4-49c01e9d5da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:57.560637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:51:57.596588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d01d74-e416-4bb9-b9f4-49c01e9d5da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:51:57.598114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d01d74-e416-4bb9-b9f4-49c01e9d5da7 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:51:57.618115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 745/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:51:59.665072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88f087-61c6-4751-8960-f724352e508c None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:59.665652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88f087-61c6-4751-8960-f724352e508c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:59.665652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88f087-61c6-4751-8960-f724352e508c None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:59.667427 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 740/1485] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:51:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:51:59.674483 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eefe5-0a64-4332-8dba-e76b1f9f5779 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:51:59.674483 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eefe5-0a64-4332-8dba-e76b1f9f5779 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:51:59.674941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eefe5-0a64-4332-8dba-e76b1f9f5779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:51:59.740046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-eb8eefe5-0a64-4332-8dba-e76b1f9f5779 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:51:59.741292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eb8eefe5-0a64-4332-8dba-e76b1f9f5779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:51:59.842997 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 746/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:51:59.881478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88f087-61c6-4751-8960-f724352e508c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:51:59.883981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9c88f087-61c6-4751-8960-f724352e508c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:52:00.001581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88f087-61c6-4751-8960-f724352e508c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:52:00.086818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:52:00.088897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238da879-4359-45d5-90b2-7ff015e0ff02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:52:00.089580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238da879-4359-45d5-90b2-7ff015e0ff02 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:52:00.089872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238da879-4359-45d5-90b2-7ff015e0ff02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:52:00.090150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238da879-4359-45d5-90b2-7ff015e0ff02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:52:00.092623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-238da879-4359-45d5-90b2-7ff015e0ff02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:52:00.352384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238da879-4359-45d5-90b2-7ff015e0ff02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:52:00.354263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238da879-4359-45d5-90b2-7ff015e0ff02 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:52:00.383769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 741/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:52:02.580927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939cbb6-f7a1-4d89-a318-ed902637ba1c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:52:02.581482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939cbb6-f7a1-4d89-a318-ed902637ba1c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:52:02.581482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939cbb6-f7a1-4d89-a318-ed902637ba1c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:52:02.582864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 747/1488] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:52:02.589014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b149c4-529b-4dfb-893d-96f1d71ce4fd None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:52:02.589474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b149c4-529b-4dfb-893d-96f1d71ce4fd None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:52:02.589570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b149c4-529b-4dfb-893d-96f1d71ce4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:52:02.660312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e0b149c4-529b-4dfb-893d-96f1d71ce4fd None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:52:02.661529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e0b149c4-529b-4dfb-893d-96f1d71ce4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:52:02.719459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:52:02.781136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 742/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:52:02.834279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939cbb6-f7a1-4d89-a318-ed902637ba1c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:52:02.836615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2939cbb6-f7a1-4d89-a318-ed902637ba1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:52:02.875627 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:52:02.958596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939cbb6-f7a1-4d89-a318-ed902637ba1c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:52:03.035893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703ab71-3dc1-45ad-b01a-bf8bf6240f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:52:03.036586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703ab71-3dc1-45ad-b01a-bf8bf6240f8a service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:52:03.036867 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703ab71-3dc1-45ad-b01a-bf8bf6240f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:52:03.037144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703ab71-3dc1-45ad-b01a-bf8bf6240f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:52:03.038791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9703ab71-3dc1-45ad-b01a-bf8bf6240f8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:52:03.126297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:52:03.160425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703ab71-3dc1-45ad-b01a-bf8bf6240f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:52:03.161918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703ab71-3dc1-45ad-b01a-bf8bf6240f8a service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:52:03.190876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 748/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:52:05.428030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd36ee-84af-4f00-a51e-24a27343c496 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:52:05.428971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd36ee-84af-4f00-a51e-24a27343c496 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:52:05.428971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd36ee-84af-4f00-a51e-24a27343c496 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:52:05.431322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 743/1491] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jul 11 20:52:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:52:05.437927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed6df8-3620-44a0-8a9c-38d6d458d46f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:52:05.439002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed6df8-3620-44a0-8a9c-38d6d458d46f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:52:05.439152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed6df8-3620-44a0-8a9c-38d6d458d46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:52:05.511814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-13ed6df8-3620-44a0-8a9c-38d6d458d46f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:52:05.513101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-13ed6df8-3620-44a0-8a9c-38d6d458d46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:52:05.620611 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 749/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jul 11 20:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:52:05.663022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd36ee-84af-4f00-a51e-24a27343c496 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:52:05.665586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-33bd36ee-84af-4f00-a51e-24a27343c496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:52:05.779217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd36ee-84af-4f00-a51e-24a27343c496 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:52:05.857958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ae39a-aa18-4701-9b03-677f01b0b0a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:52:05.858623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae39a-aa18-4701-9b03-677f01b0b0a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:52:05.858903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae39a-aa18-4701-9b03-677f01b0b0a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:52:05.859172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae39a-aa18-4701-9b03-677f01b0b0a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:52:05.860854 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-322ae39a-aa18-4701-9b03-677f01b0b0a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:52:05.869806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:52:05.984694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ae39a-aa18-4701-9b03-677f01b0b0a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:52:05.986132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ae39a-aa18-4701-9b03-677f01b0b0a3 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:52:06.006203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 744/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:52:15.618278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=82 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:52:16.003581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=50 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:39.801427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d7744-4b69-4994-82cd-0788dc79757b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:39.802042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d7744-4b69-4994-82cd-0788dc79757b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:39.802042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d7744-4b69-4994-82cd-0788dc79757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:39.891025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-dd4d7744-4b69-4994-82cd-0788dc79757b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:53:39.892280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dd4d7744-4b69-4994-82cd-0788dc79757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:39.997899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 750/1494] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:53:40.006307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df6787-53df-4b99-9a9e-6b27474dd723 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:40.006827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df6787-53df-4b99-9a9e-6b27474dd723 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:40.006827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df6787-53df-4b99-9a9e-6b27474dd723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:40.086291 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-40df6787-53df-4b99-9a9e-6b27474dd723 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:53:40.087426 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-40df6787-53df-4b99-9a9e-6b27474dd723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:40.191925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 745/1495] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:53:40.196826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d7744-4b69-4994-82cd-0788dc79757b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:53:40.198048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dd4d7744-4b69-4994-82cd-0788dc79757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:40.209899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d7744-4b69-4994-82cd-0788dc79757b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:53:40.276322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:40.314816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:53:40.315915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:40.316321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:40.316669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:40.317763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:53:40.318173 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:53:40.319782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:53:40.319782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 11 20:53:40.319782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: Jul 11 20:53:40.321120 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:53:40.334106 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 751/1496] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:53:40.342737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df6787-53df-4b99-9a9e-6b27474dd723 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:53:40.352101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df6787-53df-4b99-9a9e-6b27474dd723 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:53:40.419286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:40.458623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e009596-930d-43c5-aeb3-fe0ec2cb236d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:53:40.459290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e009596-930d-43c5-aeb3-fe0ec2cb236d admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:40.459568 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e009596-930d-43c5-aeb3-fe0ec2cb236d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:40.459837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e009596-930d-43c5-aeb3-fe0ec2cb236d admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:40.460558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e009596-930d-43c5-aeb3-fe0ec2cb236d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:53:40.462319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-4e009596-930d-43c5-aeb3-fe0ec2cb236d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:53:40.462319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Jul 11 20:53:40.462319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: Jul 11 20:53:40.463504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e009596-930d-43c5-aeb3-fe0ec2cb236d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:53:40.523300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 746/1497] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:53:40 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:53:40.527840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:53:40.529417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:40.576960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb526349-b9c3-44a4-a778-9acd4cbe818a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:53:40.675882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91603460-a217-4370-9f34-a51a3feb8420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:53:40.676588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91603460-a217-4370-9f34-a51a3feb8420 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:40.676863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91603460-a217-4370-9f34-a51a3feb8420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:40.677114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91603460-a217-4370-9f34-a51a3feb8420 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:40.677824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91603460-a217-4370-9f34-a51a3feb8420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:53:40.678840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-91603460-a217-4370-9f34-a51a3feb8420 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:53:40.678840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 11 20:53:40.678840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: Jul 11 20:53:40.680042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91603460-a217-4370-9f34-a51a3feb8420 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:53:40.724433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 752/1498] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:53:40 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:53:40.728908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e009596-930d-43c5-aeb3-fe0ec2cb236d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:53:40.730562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4e009596-930d-43c5-aeb3-fe0ec2cb236d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:40.794325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e009596-930d-43c5-aeb3-fe0ec2cb236d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:53:40.900140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:40.912126 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad73b1e-ee30-428e-8024-de2104146055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:53:40.912813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad73b1e-ee30-428e-8024-de2104146055 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:40.913091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad73b1e-ee30-428e-8024-de2104146055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:40.913356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad73b1e-ee30-428e-8024-de2104146055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:40.914084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad73b1e-ee30-428e-8024-de2104146055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:53:40.915111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-cad73b1e-ee30-428e-8024-de2104146055 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:53:40.915111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Jul 11 20:53:40.915111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: Jul 11 20:53:40.915990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad73b1e-ee30-428e-8024-de2104146055 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:53:40.928989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 747/1499] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:53:40.933546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91603460-a217-4370-9f34-a51a3feb8420 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:53:40.935060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-91603460-a217-4370-9f34-a51a3feb8420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:40.946713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91603460-a217-4370-9f34-a51a3feb8420 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:53:40.972199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:41.051809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743a875-45d1-414d-9ed5-cd39b1f50794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:53:41.052531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743a875-45d1-414d-9ed5-cd39b1f50794 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:41.052811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743a875-45d1-414d-9ed5-cd39b1f50794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:41.053078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743a875-45d1-414d-9ed5-cd39b1f50794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:41.053811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743a875-45d1-414d-9ed5-cd39b1f50794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:53:41.054850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-6743a875-45d1-414d-9ed5-cd39b1f50794 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:53:41.054850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 11 20:53:41.054850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: Jul 11 20:53:41.055735 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743a875-45d1-414d-9ed5-cd39b1f50794 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:53:41.069682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 753/1500] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:53:41.074214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad73b1e-ee30-428e-8024-de2104146055 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:53:41.075758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cad73b1e-ee30-428e-8024-de2104146055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:41.111345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad73b1e-ee30-428e-8024-de2104146055 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:53:41.212187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a44522-5ce7-4a30-abe7-f1fe9e41fc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:53:41.212999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a44522-5ce7-4a30-abe7-f1fe9e41fc6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:41.213278 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a44522-5ce7-4a30-abe7-f1fe9e41fc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:41.213549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a44522-5ce7-4a30-abe7-f1fe9e41fc6c admin admin] PATH_INFO: `/v3/projects/5539235ada92418c97610e3b7977e4d0/users/d2e2e653e2fd4e1a9e00b4eaf2b99599/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:41.240724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a44522-5ce7-4a30-abe7-f1fe9e41fc6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5539235ada92418c97610e3b7977e4d0, user_id=d2e2e653e2fd4e1a9e00b4eaf2b99599, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:53:41.241755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-04a44522-5ce7-4a30-abe7-f1fe9e41fc6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:53:41.241755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Jul 11 20:53:41.241755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: Jul 11 20:53:41.244030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a44522-5ce7-4a30-abe7-f1fe9e41fc6c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:53:41.279897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 748/1501] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:53:41 2022] PUT /identity/v3/projects/5539235ada92418c97610e3b7977e4d0/users/d2e2e653e2fd4e1a9e00b4eaf2b99599/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:53:41.296852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd6646-46f3-455f-87ab-2040fd1b8625 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:41.297145 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd6646-46f3-455f-87ab-2040fd1b8625 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:41.297486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd6646-46f3-455f-87ab-2040fd1b8625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:41.358278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1abd6646-46f3-455f-87ab-2040fd1b8625 None None] MFA Rules not processed for user `d2e2e653e2fd4e1a9e00b4eaf2b99599`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:53:41.359497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1abd6646-46f3-455f-87ab-2040fd1b8625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:41.435700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 754/1502] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:53:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:53:41.454596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7b20-5d71-404b-98f3-94f3baf70840 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:41.454882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7b20-5d71-404b-98f3-94f3baf70840 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:41.455215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7b20-5d71-404b-98f3-94f3baf70840 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:41.457352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 749/1503] 10.222.0.20 () {54 vars in 963 bytes} [Mon Jul 11 20:53:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:53:41.464649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0e5e5-3ce9-4798-8917-dc0d7201f170 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:41.464909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0e5e5-3ce9-4798-8917-dc0d7201f170 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:41.465211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0e5e5-3ce9-4798-8917-dc0d7201f170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:41.496788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:41.542876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bca0e5e5-3ce9-4798-8917-dc0d7201f170 None None] MFA Rules not processed for user `dee928cd256446cb9e000ceb6e9ef636`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:53:41.544092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bca0e5e5-3ce9-4798-8917-dc0d7201f170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:41.631004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 755/1504] 10.222.0.20 () {58 vars in 1060 bytes} [Mon Jul 11 20:53:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:53:41.639525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7b20-5d71-404b-98f3-94f3baf70840 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:53:41.641911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6d1f7b20-5d71-404b-98f3-94f3baf70840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:41.700438 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:41.758026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7b20-5d71-404b-98f3-94f3baf70840 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:53:41.835876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15c5c7-a2dd-4680-b882-535b55f45efa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:53:41.836576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c5c7-a2dd-4680-b882-535b55f45efa service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:41.836863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c5c7-a2dd-4680-b882-535b55f45efa service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:41.837137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c5c7-a2dd-4680-b882-535b55f45efa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:41.838813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5d15c5c7-a2dd-4680-b882-535b55f45efa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:41.953213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15c5c7-a2dd-4680-b882-535b55f45efa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:53:41.954660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15c5c7-a2dd-4680-b882-535b55f45efa service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:53:41.958255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:41.975048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 750/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:53:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:53:42.018000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e81f12-f7fa-40b8-aa89-6c0221cf419f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:42.018476 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e81f12-f7fa-40b8-aa89-6c0221cf419f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:42.018577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e81f12-f7fa-40b8-aa89-6c0221cf419f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:42.019836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 756/1506] 10.222.0.20 () {52 vars in 829 bytes} [Mon Jul 11 20:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:53:42.023806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15c5c7-a2dd-4680-b882-535b55f45efa service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:53:42.025081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5d15c5c7-a2dd-4680-b882-535b55f45efa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:42.129702 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15c5c7-a2dd-4680-b882-535b55f45efa service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:53:42.227948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:53:42.228867 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:42.229151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:42.229418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] PATH_INFO: `/v3/users/d2e2e653e2fd4e1a9e00b4eaf2b99599` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:42.242266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e2e653e2fd4e1a9e00b4eaf2b99599)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:53:42.243289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:53:42.243289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Jul 11 20:53:42.243289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: Jul 11 20:53:42.244533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:53:42.254690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:42.312627 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e2e653e2fd4e1a9e00b4eaf2b99599'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:53:42.312976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e2e653e2fd4e1a9e00b4eaf2b99599'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:53:42.325157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e2e653e2fd4e1a9e00b4eaf2b99599'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:53:42.325732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e2e653e2fd4e1a9e00b4eaf2b99599'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:53:42.345001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e2e653e2fd4e1a9e00b4eaf2b99599'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:53:42.362495 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 751/1507] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:53:42 2022] DELETE /identity/v3/users/d2e2e653e2fd4e1a9e00b4eaf2b99599 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:53:42.367066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069c71e-6be3-441e-9eb7-b6f6c34074c0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:42.367378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069c71e-6be3-441e-9eb7-b6f6c34074c0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:42.367657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069c71e-6be3-441e-9eb7-b6f6c34074c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:42.426182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e069c71e-6be3-441e-9eb7-b6f6c34074c0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:53:42.427453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e069c71e-6be3-441e-9eb7-b6f6c34074c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:42.511288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:42.528558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 757/1508] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:53:42.544250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:53:42.545505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:42.657370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9ebc-47c9-4e5c-a05c-e29db6f61eea admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:53:42.716922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:42.738066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c1ad-8723-4dcc-89fa-1d3509353786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:53:42.738716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c1ad-8723-4dcc-89fa-1d3509353786 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:42.738993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c1ad-8723-4dcc-89fa-1d3509353786 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:42.739262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c1ad-8723-4dcc-89fa-1d3509353786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:42.740935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bb63c1ad-8723-4dcc-89fa-1d3509353786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:42.856833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63c1ad-8723-4dcc-89fa-1d3509353786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:53:42.858284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63c1ad-8723-4dcc-89fa-1d3509353786 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:53:42.877808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 752/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:53:43.278780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069c71e-6be3-441e-9eb7-b6f6c34074c0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:53:43.280379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e069c71e-6be3-441e-9eb7-b6f6c34074c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:53:43.292969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069c71e-6be3-441e-9eb7-b6f6c34074c0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:53:43.392903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:53:43.393693 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:53:43.393973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:53:43.394250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] PATH_INFO: `/v3/projects/5539235ada92418c97610e3b7977e4d0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:53:43.404568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5539235ada92418c97610e3b7977e4d0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:53:43.405618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:53:43.405618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 11 20:53:43.405618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: Jul 11 20:53:43.406859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:53:43.466914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5539235ada92418c97610e3b7977e4d0', 'request_id': 'req-688dbc1b-4a29-4a44-bba2-6e6e00c95498'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:53:43.480326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5539235ada92418c97610e3b7977e4d0', 'request_id': 'req-688dbc1b-4a29-4a44-bba2-6e6e00c95498'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:53:43.490581 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5539235ada92418c97610e3b7977e4d0', 'request_id': 'req-688dbc1b-4a29-4a44-bba2-6e6e00c95498'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:53:43.514459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] The token cache is being invalidate because project 5539235ada92418c97610e3b7977e4d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:53:43.514745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:53:43.515228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-688dbc1b-4a29-4a44-bba2-6e6e00c95498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:53:43.516692 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 758/1510] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:53:43 2022] DELETE /identity/v3/projects/5539235ada92418c97610e3b7977e4d0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:53:52.875043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=62 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:53:53.512402 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:55:19.081698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c5270-8448-4805-979b-b7d41f64c375 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:19.082328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c5270-8448-4805-979b-b7d41f64c375 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:19.082328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c5270-8448-4805-979b-b7d41f64c375 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:19.083736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:55:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:55:19.088829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b359e-a779-4295-92fe-0f3381cd573e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:19.089303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b359e-a779-4295-92fe-0f3381cd573e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:19.089505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b359e-a779-4295-92fe-0f3381cd573e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:19.176463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1f3b359e-a779-4295-92fe-0f3381cd573e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:55:19.177678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1f3b359e-a779-4295-92fe-0f3381cd573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:19.412959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:55:19.568983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c5270-8448-4805-979b-b7d41f64c375 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:55:19.572519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-695c5270-8448-4805-979b-b7d41f64c375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:19.714062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c5270-8448-4805-979b-b7d41f64c375 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:55:19.720249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0029c-02d9-4769-803f-c5ef4120d0e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:55:19.720909 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0029c-02d9-4769-803f-c5ef4120d0e1 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:19.721199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0029c-02d9-4769-803f-c5ef4120d0e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:19.721478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0029c-02d9-4769-803f-c5ef4120d0e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:19.730794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0029c-02d9-4769-803f-c5ef4120d0e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:55:19.732430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0029c-02d9-4769-803f-c5ef4120d0e1 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:55:19.742484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 754/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:55:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:55:20.571197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b143c1-c0a3-45ff-a982-6acfcf47558b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:20.572098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b143c1-c0a3-45ff-a982-6acfcf47558b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:20.572098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b143c1-c0a3-45ff-a982-6acfcf47558b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:20.574249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:55:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:55:20.582328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884a8fe-82db-417a-b4c4-08180b7370c7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:20.582810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884a8fe-82db-417a-b4c4-08180b7370c7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:20.582900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884a8fe-82db-417a-b4c4-08180b7370c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:20.602190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1884a8fe-82db-417a-b4c4-08180b7370c7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:55:20.603381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1884a8fe-82db-417a-b4c4-08180b7370c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:20.612342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:55:20.766977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b143c1-c0a3-45ff-a982-6acfcf47558b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:55:20.784612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b143c1-c0a3-45ff-a982-6acfcf47558b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:55:20.791894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb386e93-c19c-4025-acc4-445539955ed3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:55:20.792559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386e93-c19c-4025-acc4-445539955ed3 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:20.792840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386e93-c19c-4025-acc4-445539955ed3 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:20.793112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386e93-c19c-4025-acc4-445539955ed3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:20.802264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb386e93-c19c-4025-acc4-445539955ed3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:55:20.803862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb386e93-c19c-4025-acc4-445539955ed3 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:55:20.813777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 761/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:55:21.572386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604589b1-2214-40b3-a9d1-60acbe3c2770 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:21.572664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604589b1-2214-40b3-a9d1-60acbe3c2770 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:21.572958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604589b1-2214-40b3-a9d1-60acbe3c2770 None None] PATH_INFO: `/v3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:21.574341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 756/1517] 10.222.0.33 () {54 vars in 925 bytes} [Mon Jul 11 20:55:21 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:55:21.581339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc4c48-70c3-4c57-bc23-af5bffac453e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:21.581875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc4c48-70c3-4c57-bc23-af5bffac453e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:21.581875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc4c48-70c3-4c57-bc23-af5bffac453e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:21.643028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8fcc4c48-70c3-4c57-bc23-af5bffac453e None None] MFA Rules not processed for user `946d11b552754364b55eb495f68ab852`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:55:21.644311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8fcc4c48-70c3-4c57-bc23-af5bffac453e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:21.663147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:55:21.725042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 762/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jul 11 20:55:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:55:30.596898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=56 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:55:30.905325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a43d9b-e2fb-4c70-9f2e-f2832f1fc40a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:30.905570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a43d9b-e2fb-4c70-9f2e-f2832f1fc40a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:30.905859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a43d9b-e2fb-4c70-9f2e-f2832f1fc40a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:30.907266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 757/1519] 10.222.0.20 () {54 vars in 960 bytes} [Mon Jul 11 20:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:55:30.913863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04de482-cb69-468a-8bad-b158add3de47 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:30.914368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04de482-cb69-468a-8bad-b158add3de47 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:30.914477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04de482-cb69-468a-8bad-b158add3de47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:30.945839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a04de482-cb69-468a-8bad-b158add3de47 None None] MFA Rules not processed for user `946d11b552754364b55eb495f68ab852`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:55:30.947042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a04de482-cb69-468a-8bad-b158add3de47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:30.957466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 763/1520] 10.222.0.20 () {58 vars in 1057 bytes} [Mon Jul 11 20:55:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:55:30.965451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a43d9b-e2fb-4c70-9f2e-f2832f1fc40a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:55:30.981134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a43d9b-e2fb-4c70-9f2e-f2832f1fc40a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:55:30.986575 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82c2ff-1a68-42fd-9712-490b22ffa00f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:55:30.987213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82c2ff-1a68-42fd-9712-490b22ffa00f service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:30.987496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82c2ff-1a68-42fd-9712-490b22ffa00f service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:30.987769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82c2ff-1a68-42fd-9712-490b22ffa00f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:30.996814 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82c2ff-1a68-42fd-9712-490b22ffa00f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:55:30.998403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82c2ff-1a68-42fd-9712-490b22ffa00f service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:55:31.008691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 758/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:55:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:55:32.089608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205705bd-71f4-4654-83d8-1ed09dba0b10 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:32.090114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205705bd-71f4-4654-83d8-1ed09dba0b10 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:32.090229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205705bd-71f4-4654-83d8-1ed09dba0b10 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:32.091594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:55:32.097892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23061b4-ebdf-4834-91ae-d5459dc7e03c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:32.098323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23061b4-ebdf-4834-91ae-d5459dc7e03c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:32.098434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23061b4-ebdf-4834-91ae-d5459dc7e03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:32.118057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a23061b4-ebdf-4834-91ae-d5459dc7e03c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:55:32.119261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a23061b4-ebdf-4834-91ae-d5459dc7e03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:32.127894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:55:32.292054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205705bd-71f4-4654-83d8-1ed09dba0b10 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:55:32.313815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205705bd-71f4-4654-83d8-1ed09dba0b10 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:55:32.321793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e41824-8c82-486d-9f05-ed49b79266dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:55:32.322418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e41824-8c82-486d-9f05-ed49b79266dc service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:32.322696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e41824-8c82-486d-9f05-ed49b79266dc service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:32.322977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e41824-8c82-486d-9f05-ed49b79266dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:32.331986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e41824-8c82-486d-9f05-ed49b79266dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:55:32.333683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e41824-8c82-486d-9f05-ed49b79266dc service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:55:32.343247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 765/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:55:33.173041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ae2bc-8dbf-4ce5-8375-876f548dd6dd None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:33.173561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ae2bc-8dbf-4ce5-8375-876f548dd6dd None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:33.173710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ae2bc-8dbf-4ce5-8375-876f548dd6dd None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:33.175780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 20:55:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:55:33.180849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe8fc1-e36d-43e7-a07c-db2e8c7030d3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:33.181320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe8fc1-e36d-43e7-a07c-db2e8c7030d3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:33.181476 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe8fc1-e36d-43e7-a07c-db2e8c7030d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:33.200543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-70fe8fc1-e36d-43e7-a07c-db2e8c7030d3 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:55:33.201738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-70fe8fc1-e36d-43e7-a07c-db2e8c7030d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:33.209759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:55:33.359147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ae2bc-8dbf-4ce5-8375-876f548dd6dd None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:55:33.374482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ae2bc-8dbf-4ce5-8375-876f548dd6dd None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:55:33.380219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc68e48-4920-4ed3-909e-e5e460d3c1f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:55:33.381103 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc68e48-4920-4ed3-909e-e5e460d3c1f9 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:33.381383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc68e48-4920-4ed3-909e-e5e460d3c1f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:33.381658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc68e48-4920-4ed3-909e-e5e460d3c1f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:33.390626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc68e48-4920-4ed3-909e-e5e460d3c1f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:55:33.392409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc68e48-4920-4ed3-909e-e5e460d3c1f9 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:55:33.403204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 761/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:55:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:55:37.150108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418a4bf-2219-4f20-97c6-fb390330ea0d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:37.150697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418a4bf-2219-4f20-97c6-fb390330ea0d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:37.150697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418a4bf-2219-4f20-97c6-fb390330ea0d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:37.152032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:55:37.158400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d59939-c821-460a-9d01-976b30251f2e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:37.159134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d59939-c821-460a-9d01-976b30251f2e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:37.159491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d59939-c821-460a-9d01-976b30251f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:37.183528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-00d59939-c821-460a-9d01-976b30251f2e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:55:37.185192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-00d59939-c821-460a-9d01-976b30251f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:37.193791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:55:37.269815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418a4bf-2219-4f20-97c6-fb390330ea0d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:55:37.272132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d418a4bf-2219-4f20-97c6-fb390330ea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:37.383224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418a4bf-2219-4f20-97c6-fb390330ea0d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:55:37.387870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e5d86-4afa-41b4-95bd-9f3cebdfa40f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:55:37.388553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e5d86-4afa-41b4-95bd-9f3cebdfa40f service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:37.388832 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e5d86-4afa-41b4-95bd-9f3cebdfa40f service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:37.389094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e5d86-4afa-41b4-95bd-9f3cebdfa40f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:37.399303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e5d86-4afa-41b4-95bd-9f3cebdfa40f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:55:37.400867 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e5d86-4afa-41b4-95bd-9f3cebdfa40f service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:55:37.410779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 768/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:55:39.447970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cdcd9-11a7-4872-8adb-acc3e5285037 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:39.448637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cdcd9-11a7-4872-8adb-acc3e5285037 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:39.448637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cdcd9-11a7-4872-8adb-acc3e5285037 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:39.449967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:55:39.456481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c3cd1-c680-41bd-82db-f05a70173ed0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:39.456931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c3cd1-c680-41bd-82db-f05a70173ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:39.457062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c3cd1-c680-41bd-82db-f05a70173ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:39.480224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-dc5c3cd1-c680-41bd-82db-f05a70173ed0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:55:39.482071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dc5c3cd1-c680-41bd-82db-f05a70173ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:39.493151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:55:39.530969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cdcd9-11a7-4872-8adb-acc3e5285037 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:55:39.543894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cdcd9-11a7-4872-8adb-acc3e5285037 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:55:39.548386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9b839-8880-4b6e-984d-8da2fdd8cfd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:55:39.549025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9b839-8880-4b6e-984d-8da2fdd8cfd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:39.549324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9b839-8880-4b6e-984d-8da2fdd8cfd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:39.549592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9b839-8880-4b6e-984d-8da2fdd8cfd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:39.561958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9b839-8880-4b6e-984d-8da2fdd8cfd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:55:39.563895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9b839-8880-4b6e-984d-8da2fdd8cfd4 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:55:39.574645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 764/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:55:43.096618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1348e-63d9-4b95-8b24-deae24776f06 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:43.097168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1348e-63d9-4b95-8b24-deae24776f06 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:43.097168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1348e-63d9-4b95-8b24-deae24776f06 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:43.098672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:55:43.106582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdc786-b2cf-42aa-b867-1710055b970c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:43.107369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdc786-b2cf-42aa-b867-1710055b970c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:43.107561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdc786-b2cf-42aa-b867-1710055b970c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:43.128972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-09bdc786-b2cf-42aa-b867-1710055b970c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:55:43.130243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-09bdc786-b2cf-42aa-b867-1710055b970c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:43.139687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:55:43.441512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1348e-63d9-4b95-8b24-deae24776f06 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:55:43.444159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-efc1348e-63d9-4b95-8b24-deae24776f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:43.524198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:55:43.539248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1348e-63d9-4b95-8b24-deae24776f06 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:55:43.543547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d717ff-0d2d-4bfa-b5c5-c541da968131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:55:43.544232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d717ff-0d2d-4bfa-b5c5-c541da968131 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:43.544511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d717ff-0d2d-4bfa-b5c5-c541da968131 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:43.544780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d717ff-0d2d-4bfa-b5c5-c541da968131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:43.554072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d717ff-0d2d-4bfa-b5c5-c541da968131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:55:43.555511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d717ff-0d2d-4bfa-b5c5-c541da968131 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:55:43.565400 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 771/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:55:45.617890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e197a-e958-4c3f-b9de-52c1ed004ca3 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:45.618471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e197a-e958-4c3f-b9de-52c1ed004ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:45.618471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e197a-e958-4c3f-b9de-52c1ed004ca3 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:45.619942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 20:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:55:45.626133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc69524-f8e9-4155-b8b7-fe1db81ea924 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:45.626572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc69524-f8e9-4155-b8b7-fe1db81ea924 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:45.626711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc69524-f8e9-4155-b8b7-fe1db81ea924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:45.649338 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-edc69524-f8e9-4155-b8b7-fe1db81ea924 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:55:45.651089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-edc69524-f8e9-4155-b8b7-fe1db81ea924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:45.663912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 20:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:55:45.971345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e197a-e958-4c3f-b9de-52c1ed004ca3 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:55:45.974516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ef8e197a-e958-4c3f-b9de-52c1ed004ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:55:45.988680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e197a-e958-4c3f-b9de-52c1ed004ca3 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:55:45.992405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d521910-18b8-4ad7-a492-8e24242adee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:55:45.993049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d521910-18b8-4ad7-a492-8e24242adee5 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:55:45.993325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d521910-18b8-4ad7-a492-8e24242adee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:55:45.993595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d521910-18b8-4ad7-a492-8e24242adee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:55:46.002801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d521910-18b8-4ad7-a492-8e24242adee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:55:46.004256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d521910-18b8-4ad7-a492-8e24242adee5 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:55:46.014107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 767/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:55:55.641661 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=11 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:55:56.009806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=33 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:14.895248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db226c-97e7-49ce-a82a-2644b803b7d5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:14.895933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db226c-97e7-49ce-a82a-2644b803b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:14.895933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db226c-97e7-49ce-a82a-2644b803b7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:14.929426 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-90db226c-97e7-49ce-a82a-2644b803b7d5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:14.930678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-90db226c-97e7-49ce-a82a-2644b803b7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:14.947518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 773/1540] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:57:14.957671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff8d8b-d918-4472-84db-1cababc6f995 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:14.958289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff8d8b-d918-4472-84db-1cababc6f995 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:14.958289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff8d8b-d918-4472-84db-1cababc6f995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:14.986975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1fff8d8b-d918-4472-84db-1cababc6f995 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:14.988494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1fff8d8b-d918-4472-84db-1cababc6f995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:14.998971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 768/1541] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.004045 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db226c-97e7-49ce-a82a-2644b803b7d5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.017987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db226c-97e7-49ce-a82a-2644b803b7d5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.029079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bd774-03bc-403c-8b04-126a5747751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.029774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bd774-03bc-403c-8b04-126a5747751a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.030061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bd774-03bc-403c-8b04-126a5747751a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.030326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bd774-03bc-403c-8b04-126a5747751a admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.031066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bd774-03bc-403c-8b04-126a5747751a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:15.031433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bd774-03bc-403c-8b04-126a5747751a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.033090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bd774-03bc-403c-8b04-126a5747751a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.052667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 774/1542] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:15.061402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff8d8b-d918-4472-84db-1cababc6f995 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.078783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff8d8b-d918-4472-84db-1cababc6f995 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.087205 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878f597-9bc6-4fe1-b67e-6f63d7f3b857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.087932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878f597-9bc6-4fe1-b67e-6f63d7f3b857 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.088242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878f597-9bc6-4fe1-b67e-6f63d7f3b857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.088519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878f597-9bc6-4fe1-b67e-6f63d7f3b857 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.089289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878f597-9bc6-4fe1-b67e-6f63d7f3b857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.091293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878f597-9bc6-4fe1-b67e-6f63d7f3b857 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.155186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 769/1543] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:15.159464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bd774-03bc-403c-8b04-126a5747751a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.164162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14330f75-e9fb-45ff-8991-2cc78e65ed45 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.164459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14330f75-e9fb-45ff-8991-2cc78e65ed45 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.164732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14330f75-e9fb-45ff-8991-2cc78e65ed45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.171972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bd774-03bc-403c-8b04-126a5747751a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.193139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-14330f75-e9fb-45ff-8991-2cc78e65ed45 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.194584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-14330f75-e9fb-45ff-8991-2cc78e65ed45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.264813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf76a7-9a76-42df-903b-b68ac510ba94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.265438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf76a7-9a76-42df-903b-b68ac510ba94 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.265723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf76a7-9a76-42df-903b-b68ac510ba94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.266000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf76a7-9a76-42df-903b-b68ac510ba94 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.267478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf76a7-9a76-42df-903b-b68ac510ba94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.269684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf76a7-9a76-42df-903b-b68ac510ba94 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.305510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 775/1544] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/users => generated 297 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:15.306866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31265a6f-4ee4-4cd9-b841-3ed8d2770cb9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.307154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31265a6f-4ee4-4cd9-b841-3ed8d2770cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.307439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31265a6f-4ee4-4cd9-b841-3ed8d2770cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.316877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 770/1545] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:57:15.317858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14330f75-e9fb-45ff-8991-2cc78e65ed45 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.343432 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14330f75-e9fb-45ff-8991-2cc78e65ed45 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.351371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-31265a6f-4ee4-4cd9-b841-3ed8d2770cb9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.352608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-31265a6f-4ee4-4cd9-b841-3ed8d2770cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.355471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf666-276b-4f61-9caa-e02ee672797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.360348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf666-276b-4f61-9caa-e02ee672797e admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.360665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf666-276b-4f61-9caa-e02ee672797e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.360947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf666-276b-4f61-9caa-e02ee672797e admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.361698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cf666-276b-4f61-9caa-e02ee672797e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.363364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cf666-276b-4f61-9caa-e02ee672797e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.381960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 776/1546] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.385840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 771/1547] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:15.386834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b677fb4-2e83-4727-b368-d21a7a031c34 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.387119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b677fb4-2e83-4727-b368-d21a7a031c34 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.387388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b677fb4-2e83-4727-b368-d21a7a031c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.393739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf666-276b-4f61-9caa-e02ee672797e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.412540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf666-276b-4f61-9caa-e02ee672797e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.427461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298f4a9-fd40-4069-91ce-55e47b106fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.428585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298f4a9-fd40-4069-91ce-55e47b106fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.429091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298f4a9-fd40-4069-91ce-55e47b106fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.429578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298f4a9-fd40-4069-91ce-55e47b106fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.430769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298f4a9-fd40-4069-91ce-55e47b106fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.431222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4b677fb4-2e83-4727-b368-d21a7a031c34 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.440131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298f4a9-fd40-4069-91ce-55e47b106fdc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.442200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4b677fb4-2e83-4727-b368-d21a7a031c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.460527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 772/1548] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:15.462496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dade1-2c35-475d-be57-53cabd3782d5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.463040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dade1-2c35-475d-be57-53cabd3782d5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.463517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dade1-2c35-475d-be57-53cabd3782d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.468001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 777/1549] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.468463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f520b91-7192-46d3-aba3-69f6ca365f26 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.470272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f520b91-7192-46d3-aba3-69f6ca365f26 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.470607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f520b91-7192-46d3-aba3-69f6ca365f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.497335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b63dade1-2c35-475d-be57-53cabd3782d5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.499009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b63dade1-2c35-475d-be57-53cabd3782d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.509011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 773/1550] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.513733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f240bc2-a9ee-4277-95bf-923c558cf4a2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.513733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f240bc2-a9ee-4277-95bf-923c558cf4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.513733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f240bc2-a9ee-4277-95bf-923c558cf4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.514447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7f520b91-7192-46d3-aba3-69f6ca365f26 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.515631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7f520b91-7192-46d3-aba3-69f6ca365f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.524643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 778/1551] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.525527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f520b91-7192-46d3-aba3-69f6ca365f26 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.535615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f520b91-7192-46d3-aba3-69f6ca365f26 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.537821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5f240bc2-a9ee-4277-95bf-923c558cf4a2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.539595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5f240bc2-a9ee-4277-95bf-923c558cf4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.540666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede17af-1749-4e3d-af5f-b22b3f78b628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.541630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede17af-1749-4e3d-af5f-b22b3f78b628 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.542035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede17af-1749-4e3d-af5f-b22b3f78b628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.542311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede17af-1749-4e3d-af5f-b22b3f78b628 admin admin] PATH_INFO: `/v3/projects/c6f21a94f61e42c9b967e673f02eb270/users/aa7e6c5842a84e91ac94c1107ad29158/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.551064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 774/1552] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.552334 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f240bc2-a9ee-4277-95bf-923c558cf4a2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.558593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede17af-1749-4e3d-af5f-b22b3f78b628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f21a94f61e42c9b967e673f02eb270, user_id=aa7e6c5842a84e91ac94c1107ad29158, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.559726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-3ede17af-1749-4e3d-af5f-b22b3f78b628 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:57:15.559726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 11 20:57:15.559726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: Jul 11 20:57:15.561702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede17af-1749-4e3d-af5f-b22b3f78b628 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.562919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f240bc2-a9ee-4277-95bf-923c558cf4a2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.568760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583ee65-2ac4-4651-8d2b-769bfdf46cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.569487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583ee65-2ac4-4651-8d2b-769bfdf46cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.569765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583ee65-2ac4-4651-8d2b-769bfdf46cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.570034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583ee65-2ac4-4651-8d2b-769bfdf46cb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.570770 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583ee65-2ac4-4651-8d2b-769bfdf46cb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:15.571114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583ee65-2ac4-4651-8d2b-769bfdf46cb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.572349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-2583ee65-2ac4-4651-8d2b-769bfdf46cb7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:57:15.572349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Jul 11 20:57:15.572349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: Jul 11 20:57:15.573226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583ee65-2ac4-4651-8d2b-769bfdf46cb7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.577962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 779/1553] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:15 2022] PUT /identity/v3/projects/c6f21a94f61e42c9b967e673f02eb270/users/aa7e6c5842a84e91ac94c1107ad29158/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:15.579474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bcdfc-68cf-481c-b1e9-de83b9e7a00c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.579927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bcdfc-68cf-481c-b1e9-de83b9e7a00c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.580447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bcdfc-68cf-481c-b1e9-de83b9e7a00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.586171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 775/1554] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:15.587685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d555aa-1549-4de7-8c83-ca8aab7f0e12 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.588223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d555aa-1549-4de7-8c83-ca8aab7f0e12 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.588527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d555aa-1549-4de7-8c83-ca8aab7f0e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.608034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c33bcdfc-68cf-481c-b1e9-de83b9e7a00c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.608799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c0d555aa-1549-4de7-8c83-ca8aab7f0e12 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.609293 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c33bcdfc-68cf-481c-b1e9-de83b9e7a00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.609921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c0d555aa-1549-4de7-8c83-ca8aab7f0e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.670113 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 776/1555] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.670974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d555aa-1549-4de7-8c83-ca8aab7f0e12 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.676161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 780/1556] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.676161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6ee60-065b-4dd4-8258-5a0aa0cfe883 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.676161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6ee60-065b-4dd4-8258-5a0aa0cfe883 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.676161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6ee60-065b-4dd4-8258-5a0aa0cfe883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.683747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d555aa-1549-4de7-8c83-ca8aab7f0e12 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.688506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c08b8-8cef-4456-b977-5dcda0965708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.689182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c08b8-8cef-4456-b977-5dcda0965708 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.689454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c08b8-8cef-4456-b977-5dcda0965708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.689725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c08b8-8cef-4456-b977-5dcda0965708 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.690452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c08b8-8cef-4456-b977-5dcda0965708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:15.690756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c08b8-8cef-4456-b977-5dcda0965708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.692663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c08b8-8cef-4456-b977-5dcda0965708 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.694377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6ae6ee60-065b-4dd4-8258-5a0aa0cfe883 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.695570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6ae6ee60-065b-4dd4-8258-5a0aa0cfe883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.703706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 781/1557] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.711499 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 777/1558] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:15.711676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c416a57-21e3-47d2-98e7-13fa0d44acaa None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.711676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c416a57-21e3-47d2-98e7-13fa0d44acaa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.711874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479d154-dffd-4d0b-a62b-cd50caab05d2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.711971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c416a57-21e3-47d2-98e7-13fa0d44acaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.712058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479d154-dffd-4d0b-a62b-cd50caab05d2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.712058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479d154-dffd-4d0b-a62b-cd50caab05d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.716395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:15.724884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5c416a57-21e3-47d2-98e7-13fa0d44acaa None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.726131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5c416a57-21e3-47d2-98e7-13fa0d44acaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.729019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a479d154-dffd-4d0b-a62b-cd50caab05d2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.730180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a479d154-dffd-4d0b-a62b-cd50caab05d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.734200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 782/1559] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.737384 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e6383-5965-4008-b65b-e04ee49efd81 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.737672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e6383-5965-4008-b65b-e04ee49efd81 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.737942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e6383-5965-4008-b65b-e04ee49efd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.738791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 778/1560] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.739535 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479d154-dffd-4d0b-a62b-cd50caab05d2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.750297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479d154-dffd-4d0b-a62b-cd50caab05d2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.755383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0cbeb-473b-429f-b045-9f3192dc2858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.756038 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0cbeb-473b-429f-b045-9f3192dc2858 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.756359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0cbeb-473b-429f-b045-9f3192dc2858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.756634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0cbeb-473b-429f-b045-9f3192dc2858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.757321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0cbeb-473b-429f-b045-9f3192dc2858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.757415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8c9e6383-5965-4008-b65b-e04ee49efd81 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.758588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c9e6383-5965-4008-b65b-e04ee49efd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.759222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0cbeb-473b-429f-b045-9f3192dc2858 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.766561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 783/1561] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.767788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a67e0f-9ee7-4d4a-b97e-417a61d7dd27 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.768081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a67e0f-9ee7-4d4a-b97e-417a61d7dd27 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.768449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a67e0f-9ee7-4d4a-b97e-417a61d7dd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.782896 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:15.788671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-88a67e0f-9ee7-4d4a-b97e-417a61d7dd27 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.790656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-88a67e0f-9ee7-4d4a-b97e-417a61d7dd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.791102 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 779/1562] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:15.792358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a2ebe-e91d-47c4-9278-ac9fad61dc7f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.792649 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a2ebe-e91d-47c4-9278-ac9fad61dc7f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.792926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a2ebe-e91d-47c4-9278-ac9fad61dc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.810962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bc8a2ebe-e91d-47c4-9278-ac9fad61dc7f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.812238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bc8a2ebe-e91d-47c4-9278-ac9fad61dc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.858492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 784/1563] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.859591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a67e0f-9ee7-4d4a-b97e-417a61d7dd27 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.868870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a67e0f-9ee7-4d4a-b97e-417a61d7dd27 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.873779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288510a4-3c69-4c2b-9aa2-a835daca3a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.874442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288510a4-3c69-4c2b-9aa2-a835daca3a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.874720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288510a4-3c69-4c2b-9aa2-a835daca3a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.875123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288510a4-3c69-4c2b-9aa2-a835daca3a59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.875863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288510a4-3c69-4c2b-9aa2-a835daca3a59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:15.876199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288510a4-3c69-4c2b-9aa2-a835daca3a59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.877827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288510a4-3c69-4c2b-9aa2-a835daca3a59 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.884584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 780/1564] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:15.885993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a2ebe-e91d-47c4-9278-ac9fad61dc7f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.890084 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 785/1565] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:15.891079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288510a4-3c69-4c2b-9aa2-a835daca3a59 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.896136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a2ebe-e91d-47c4-9278-ac9fad61dc7f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.901432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288510a4-3c69-4c2b-9aa2-a835daca3a59 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.901982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab2af9-4760-472f-8744-348f9f01788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.902352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab2af9-4760-472f-8744-348f9f01788d admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.902642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab2af9-4760-472f-8744-348f9f01788d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.902918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab2af9-4760-472f-8744-348f9f01788d admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.903617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab2af9-4760-472f-8744-348f9f01788d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.905588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab2af9-4760-472f-8744-348f9f01788d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.906220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd25cba-89d6-4557-8100-5ec027150f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.906866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd25cba-89d6-4557-8100-5ec027150f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.907146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd25cba-89d6-4557-8100-5ec027150f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.907416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd25cba-89d6-4557-8100-5ec027150f74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.908152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd25cba-89d6-4557-8100-5ec027150f74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:15.908461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd25cba-89d6-4557-8100-5ec027150f74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:15.910092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd25cba-89d6-4557-8100-5ec027150f74 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:15.922695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 786/1566] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:15.924006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff33a9c-70a5-4925-9822-27218434b415 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.924356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff33a9c-70a5-4925-9822-27218434b415 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.924596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff33a9c-70a5-4925-9822-27218434b415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:15.933531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 781/1567] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:15.934557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab2af9-4760-472f-8744-348f9f01788d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:15.944277 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6ff33a9c-70a5-4925-9822-27218434b415 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:15.945527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6ff33a9c-70a5-4925-9822-27218434b415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:15.991624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab2af9-4760-472f-8744-348f9f01788d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:15.995463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1c7ab-c655-4bad-9a3c-660a2d5b6bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:15.996124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c7ab-c655-4bad-9a3c-660a2d5b6bdf service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:15.996425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c7ab-c655-4bad-9a3c-660a2d5b6bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:15.996693 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c7ab-c655-4bad-9a3c-660a2d5b6bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.004350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 787/1568] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:16.005156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff33a9c-70a5-4925-9822-27218434b415 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.014399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff33a9c-70a5-4925-9822-27218434b415 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.019156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5865-e735-402a-b467-44bd836dfb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.019789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5865-e735-402a-b467-44bd836dfb8d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.020290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5865-e735-402a-b467-44bd836dfb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.020290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5865-e735-402a-b467-44bd836dfb8d admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.021026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef5865-e735-402a-b467-44bd836dfb8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.022953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef5865-e735-402a-b467-44bd836dfb8d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.043267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 788/1569] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/users => generated 286 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.044198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5865-e735-402a-b467-44bd836dfb8d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.054180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5865-e735-402a-b467-44bd836dfb8d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.057293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1c7ab-c655-4bad-9a3c-660a2d5b6bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.059148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1c7ab-c655-4bad-9a3c-660a2d5b6bdf service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.059288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d5371-c0f4-464f-b93d-517b1a91b603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.059854 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d5371-c0f4-464f-b93d-517b1a91b603 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.060158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d5371-c0f4-464f-b93d-517b1a91b603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.060435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d5371-c0f4-464f-b93d-517b1a91b603 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.061162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d5371-c0f4-464f-b93d-517b1a91b603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:16.061466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d5371-c0f4-464f-b93d-517b1a91b603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.063051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d5371-c0f4-464f-b93d-517b1a91b603 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.069466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 782/1570] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:16.070418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1c7ab-c655-4bad-9a3c-660a2d5b6bdf service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.074962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:16.076463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 789/1571] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:16.077366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d5371-c0f4-464f-b93d-517b1a91b603 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.087698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1c7ab-c655-4bad-9a3c-660a2d5b6bdf service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.088091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d5371-c0f4-464f-b93d-517b1a91b603 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.092926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f419717-9297-419b-833d-e06cbaf23975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.093116 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da5532-9c41-4824-a96c-fcc1a34c4289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.093619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f419717-9297-419b-833d-e06cbaf23975 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.093739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da5532-9c41-4824-a96c-fcc1a34c4289 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.093911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f419717-9297-419b-833d-e06cbaf23975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.094009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da5532-9c41-4824-a96c-fcc1a34c4289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.094164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f419717-9297-419b-833d-e06cbaf23975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.094292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da5532-9c41-4824-a96c-fcc1a34c4289 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.095012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da5532-9c41-4824-a96c-fcc1a34c4289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:16.095124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f419717-9297-419b-833d-e06cbaf23975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.095322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da5532-9c41-4824-a96c-fcc1a34c4289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.096236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-9f419717-9297-419b-833d-e06cbaf23975 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:57:16.096236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 11 20:57:16.096236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: Jul 11 20:57:16.097005 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da5532-9c41-4824-a96c-fcc1a34c4289 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.097483 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f419717-9297-419b-833d-e06cbaf23975 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.110906 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 783/1572] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:16.111770 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da5532-9c41-4824-a96c-fcc1a34c4289 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.122998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da5532-9c41-4824-a96c-fcc1a34c4289 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.128682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b41242-a5d1-41f8-b8f3-acfda7788711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.129670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b41242-a5d1-41f8-b8f3-acfda7788711 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.130148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b41242-a5d1-41f8-b8f3-acfda7788711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.130394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b41242-a5d1-41f8-b8f3-acfda7788711 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.131150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b41242-a5d1-41f8-b8f3-acfda7788711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.133171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b41242-a5d1-41f8-b8f3-acfda7788711 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.140295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 790/1573] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.140295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f419717-9297-419b-833d-e06cbaf23975 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.170616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 784/1574] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.171492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b41242-a5d1-41f8-b8f3-acfda7788711 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.175242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:16.214920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f419717-9297-419b-833d-e06cbaf23975 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.247658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b41242-a5d1-41f8-b8f3-acfda7788711 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.254181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3dcc3f-f173-47cd-8e23-5eba04407372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.254878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dcc3f-f173-47cd-8e23-5eba04407372 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.255383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dcc3f-f173-47cd-8e23-5eba04407372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.255501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dcc3f-f173-47cd-8e23-5eba04407372 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.256174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3dcc3f-f173-47cd-8e23-5eba04407372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:16.256519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3dcc3f-f173-47cd-8e23-5eba04407372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.258243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3dcc3f-f173-47cd-8e23-5eba04407372 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.274942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 785/1575] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:16.274942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3dcc3f-f173-47cd-8e23-5eba04407372 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.285551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7257a7-6f2c-4aa9-ac7c-280834c6ddd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.286305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7257a7-6f2c-4aa9-ac7c-280834c6ddd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.286616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7257a7-6f2c-4aa9-ac7c-280834c6ddd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.286840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7257a7-6f2c-4aa9-ac7c-280834c6ddd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.287539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7257a7-6f2c-4aa9-ac7c-280834c6ddd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.288485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3dcc3f-f173-47cd-8e23-5eba04407372 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.289531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7257a7-6f2c-4aa9-ac7c-280834c6ddd2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.293377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347a0ee-7647-47e8-b02c-5a01360dad6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.294111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a0ee-7647-47e8-b02c-5a01360dad6f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.294452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a0ee-7647-47e8-b02c-5a01360dad6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.294779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a0ee-7647-47e8-b02c-5a01360dad6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.295564 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347a0ee-7647-47e8-b02c-5a01360dad6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.297246 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347a0ee-7647-47e8-b02c-5a01360dad6f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.308753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 786/1576] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.309601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347a0ee-7647-47e8-b02c-5a01360dad6f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.315974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 791/1577] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.317106 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7257a7-6f2c-4aa9-ac7c-280834c6ddd2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.320193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347a0ee-7647-47e8-b02c-5a01360dad6f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.325090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7c97f-4909-4d23-9790-e719838e3ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.326176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7c97f-4909-4d23-9790-e719838e3ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.326176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7c97f-4909-4d23-9790-e719838e3ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.326377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7c97f-4909-4d23-9790-e719838e3ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.327256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7c97f-4909-4d23-9790-e719838e3ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.328966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7c97f-4909-4d23-9790-e719838e3ed3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.330686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7257a7-6f2c-4aa9-ac7c-280834c6ddd2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.335533 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d9db7-56b6-4098-97a0-67c649587da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.336303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d9db7-56b6-4098-97a0-67c649587da7 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.336645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d9db7-56b6-4098-97a0-67c649587da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.336987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d9db7-56b6-4098-97a0-67c649587da7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.337739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d9db7-56b6-4098-97a0-67c649587da7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.339705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d9db7-56b6-4098-97a0-67c649587da7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.358321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 787/1578] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.362135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7c97f-4909-4d23-9790-e719838e3ed3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.380942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 792/1579] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/projects => generated 355 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.381786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d9db7-56b6-4098-97a0-67c649587da7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.449678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d9db7-56b6-4098-97a0-67c649587da7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.449842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7c97f-4909-4d23-9790-e719838e3ed3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.454674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6342f-d2a4-43b7-be79-78b855cba3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.454838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564fae3-d226-4238-a71a-f76b7c566e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.455400 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6342f-d2a4-43b7-be79-78b855cba3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.455508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564fae3-d226-4238-a71a-f76b7c566e78 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.455685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6342f-d2a4-43b7-be79-78b855cba3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.455793 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564fae3-d226-4238-a71a-f76b7c566e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.455983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564fae3-d226-4238-a71a-f76b7c566e78 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.456089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6342f-d2a4-43b7-be79-78b855cba3b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.457350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6342f-d2a4-43b7-be79-78b855cba3b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.457465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564fae3-d226-4238-a71a-f76b7c566e78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.458912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6342f-d2a4-43b7-be79-78b855cba3b7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.459429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-d564fae3-d226-4238-a71a-f76b7c566e78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:57:16.459429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Jul 11 20:57:16.459429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: Jul 11 20:57:16.460997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564fae3-d226-4238-a71a-f76b7c566e78 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.483130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 793/1580] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.483794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6342f-d2a4-43b7-be79-78b855cba3b7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.489193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 788/1581] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.490006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564fae3-d226-4238-a71a-f76b7c566e78 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.496315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6342f-d2a4-43b7-be79-78b855cba3b7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.501115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564fae3-d226-4238-a71a-f76b7c566e78 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.502794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b55670-5ee0-41d5-bc69-f1469e49b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.503744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b55670-5ee0-41d5-bc69-f1469e49b7ff admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.504240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b55670-5ee0-41d5-bc69-f1469e49b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.504680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b55670-5ee0-41d5-bc69-f1469e49b7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.505551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b55670-5ee0-41d5-bc69-f1469e49b7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.506221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e31c40-3fb3-4097-be6c-fa2881ac557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.506988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e31c40-3fb3-4097-be6c-fa2881ac557a admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.507584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b55670-5ee0-41d5-bc69-f1469e49b7ff admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.507714 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e31c40-3fb3-4097-be6c-fa2881ac557a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.508028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e31c40-3fb3-4097-be6c-fa2881ac557a admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.509169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e31c40-3fb3-4097-be6c-fa2881ac557a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.511671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e31c40-3fb3-4097-be6c-fa2881ac557a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.524098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 789/1582] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.525197 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e31c40-3fb3-4097-be6c-fa2881ac557a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.539550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e31c40-3fb3-4097-be6c-fa2881ac557a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.546687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d23d10-fdc1-4cb1-87d0-133236fef361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.547476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d23d10-fdc1-4cb1-87d0-133236fef361 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.547977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d23d10-fdc1-4cb1-87d0-133236fef361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.548397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d23d10-fdc1-4cb1-87d0-133236fef361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.549232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d23d10-fdc1-4cb1-87d0-133236fef361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.551071 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d23d10-fdc1-4cb1-87d0-133236fef361 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.552946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:16.560264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 794/1583] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/projects => generated 389 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.562339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b55670-5ee0-41d5-bc69-f1469e49b7ff admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.569088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 790/1584] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.570099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d23d10-fdc1-4cb1-87d0-133236fef361 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.579358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:16.583246 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d23d10-fdc1-4cb1-87d0-133236fef361 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.648400 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b55670-5ee0-41d5-bc69-f1469e49b7ff admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.654088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c14cda-f2b1-4876-9021-97c1879bbc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.654870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c14cda-f2b1-4876-9021-97c1879bbc58 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.655222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c14cda-f2b1-4876-9021-97c1879bbc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.655495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c14cda-f2b1-4876-9021-97c1879bbc58 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.656227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c14cda-f2b1-4876-9021-97c1879bbc58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.658171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c14cda-f2b1-4876-9021-97c1879bbc58 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.658910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7f9b7-686f-4bbc-b5df-ba7c0fba3556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.660058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7f9b7-686f-4bbc-b5df-ba7c0fba3556 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.660532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7f9b7-686f-4bbc-b5df-ba7c0fba3556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.661456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7f9b7-686f-4bbc-b5df-ba7c0fba3556 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.662312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7f9b7-686f-4bbc-b5df-ba7c0fba3556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.664651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7f9b7-686f-4bbc-b5df-ba7c0fba3556 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.682258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 795/1585] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.684556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c14cda-f2b1-4876-9021-97c1879bbc58 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.689505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 791/1586] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/users => generated 278 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.690242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7f9b7-686f-4bbc-b5df-ba7c0fba3556 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.699551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c14cda-f2b1-4876-9021-97c1879bbc58 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.700398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7f9b7-686f-4bbc-b5df-ba7c0fba3556 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.704618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc26145-5a66-42aa-8df2-d201a83ec69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.704978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d6f91-04e2-43b9-90fb-ba286fb93ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.705509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc26145-5a66-42aa-8df2-d201a83ec69c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.705648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d6f91-04e2-43b9-90fb-ba286fb93ddd admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.705934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d6f91-04e2-43b9-90fb-ba286fb93ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.706042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc26145-5a66-42aa-8df2-d201a83ec69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.706205 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d6f91-04e2-43b9-90fb-ba286fb93ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.706463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc26145-5a66-42aa-8df2-d201a83ec69c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.707037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d6f91-04e2-43b9-90fb-ba286fb93ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.707550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc26145-5a66-42aa-8df2-d201a83ec69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.708678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d6f91-04e2-43b9-90fb-ba286fb93ddd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.709409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc26145-5a66-42aa-8df2-d201a83ec69c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.719540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 792/1587] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.719540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d6f91-04e2-43b9-90fb-ba286fb93ddd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.723043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 796/1588] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.723043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc26145-5a66-42aa-8df2-d201a83ec69c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.732706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d6f91-04e2-43b9-90fb-ba286fb93ddd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.736385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc26145-5a66-42aa-8df2-d201a83ec69c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.739104 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1e1ef-bc7b-400c-aa8a-5e79241fd5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.739999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1e1ef-bc7b-400c-aa8a-5e79241fd5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.740351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1e1ef-bc7b-400c-aa8a-5e79241fd5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.740661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1e1ef-bc7b-400c-aa8a-5e79241fd5d7 admin admin] PATH_INFO: `/v3/projects/7dbd5e8bc3f74241ae033e620f2f9c90/users/bad8ee8b4344457b861429ca235c824f/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.743802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad3fc6-7a2b-4f2d-a886-80c097230498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.745170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad3fc6-7a2b-4f2d-a886-80c097230498 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.745599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad3fc6-7a2b-4f2d-a886-80c097230498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.745883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad3fc6-7a2b-4f2d-a886-80c097230498 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.746669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad3fc6-7a2b-4f2d-a886-80c097230498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.748764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad3fc6-7a2b-4f2d-a886-80c097230498 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.756687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1e1ef-bc7b-400c-aa8a-5e79241fd5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dbd5e8bc3f74241ae033e620f2f9c90, user_id=bad8ee8b4344457b861429ca235c824f, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.759414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1e1ef-bc7b-400c-aa8a-5e79241fd5d7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.772253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 797/1589] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/users => generated 295 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:16.773530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad3fc6-7a2b-4f2d-a886-80c097230498 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.774267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 793/1590] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:16 2022] PUT /identity/v3/projects/7dbd5e8bc3f74241ae033e620f2f9c90/users/bad8ee8b4344457b861429ca235c824f/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:16.775450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1e1ef-bc7b-400c-aa8a-5e79241fd5d7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.786644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad3fc6-7a2b-4f2d-a886-80c097230498 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.848699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1e1ef-bc7b-400c-aa8a-5e79241fd5d7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.855615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c2a7a-2b51-4328-80f7-2a7a6b403a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.856420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2a7a-2b51-4328-80f7-2a7a6b403a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.856766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2a7a-2b51-4328-80f7-2a7a6b403a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.857560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2a7a-2b51-4328-80f7-2a7a6b403a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.858151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c2a7a-2b51-4328-80f7-2a7a6b403a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.860022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c2a7a-2b51-4328-80f7-2a7a6b403a47 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.864948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b2a2a-691b-4974-8dd6-bde571dfce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.865649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b2a2a-691b-4974-8dd6-bde571dfce12 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.865931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b2a2a-691b-4974-8dd6-bde571dfce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.866205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b2a2a-691b-4974-8dd6-bde571dfce12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.867023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b2a2a-691b-4974-8dd6-bde571dfce12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.873087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b2a2a-691b-4974-8dd6-bde571dfce12 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.873819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 794/1591] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.874748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c2a7a-2b51-4328-80f7-2a7a6b403a47 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.887233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c2a7a-2b51-4328-80f7-2a7a6b403a47 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.893741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 798/1592] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.894815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b2a2a-691b-4974-8dd6-bde571dfce12 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.896261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e1985-c98d-45f4-aaaf-d3824b2ee0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.897148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e1985-c98d-45f4-aaaf-d3824b2ee0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.897482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e1985-c98d-45f4-aaaf-d3824b2ee0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.897897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e1985-c98d-45f4-aaaf-d3824b2ee0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.898829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e1985-c98d-45f4-aaaf-d3824b2ee0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.900725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e1985-c98d-45f4-aaaf-d3824b2ee0d2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.908186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b2a2a-691b-4974-8dd6-bde571dfce12 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.914772 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745945e5-b567-48fa-a304-afd6f48ff70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.915840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745945e5-b567-48fa-a304-afd6f48ff70d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.916320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745945e5-b567-48fa-a304-afd6f48ff70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.916888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 795/1593] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.917895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e1985-c98d-45f4-aaaf-d3824b2ee0d2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.919910 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745945e5-b567-48fa-a304-afd6f48ff70d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.920905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745945e5-b567-48fa-a304-afd6f48ff70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.922738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745945e5-b567-48fa-a304-afd6f48ff70d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.931100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e1985-c98d-45f4-aaaf-d3824b2ee0d2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.937045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884abbf-1ba5-421c-a6f9-7f6c40ac70ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.937719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884abbf-1ba5-421c-a6f9-7f6c40ac70ef admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.937997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884abbf-1ba5-421c-a6f9-7f6c40ac70ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.938288 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884abbf-1ba5-421c-a6f9-7f6c40ac70ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.938660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 799/1594] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.939675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745945e5-b567-48fa-a304-afd6f48ff70d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.942147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884abbf-1ba5-421c-a6f9-7f6c40ac70ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.943986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884abbf-1ba5-421c-a6f9-7f6c40ac70ef admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.951585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745945e5-b567-48fa-a304-afd6f48ff70d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:16.957043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 796/1595] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.959203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa72e3-2e18-43f0-919b-779ea99abd5a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.959484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa72e3-2e18-43f0-919b-779ea99abd5a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.959760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa72e3-2e18-43f0-919b-779ea99abd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.965171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc048e3a-46a2-499d-a934-96d8c26d4762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:16.966002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048e3a-46a2-499d-a934-96d8c26d4762 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:16.966288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048e3a-46a2-499d-a934-96d8c26d4762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:16.966657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048e3a-46a2-499d-a934-96d8c26d4762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:16.967282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc048e3a-46a2-499d-a934-96d8c26d4762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:16.968967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc048e3a-46a2-499d-a934-96d8c26d4762 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:16.981903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 800/1596] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:16.982660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc048e3a-46a2-499d-a934-96d8c26d4762 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:16.993874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d6aa72e3-2e18-43f0-919b-779ea99abd5a None None] MFA Rules not processed for user `bad8ee8b4344457b861429ca235c824f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:16.995409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d6aa72e3-2e18-43f0-919b-779ea99abd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:17.003526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc048e3a-46a2-499d-a934-96d8c26d4762 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.013288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cabee-0f9e-439a-9f6a-a23ad34b547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.015322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:17.015727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cabee-0f9e-439a-9f6a-a23ad34b547e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.016143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cabee-0f9e-439a-9f6a-a23ad34b547e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.017401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cabee-0f9e-439a-9f6a-a23ad34b547e admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.022556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cabee-0f9e-439a-9f6a-a23ad34b547e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.024347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cabee-0f9e-439a-9f6a-a23ad34b547e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.036403 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 801/1597] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:17.037436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cabee-0f9e-439a-9f6a-a23ad34b547e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.054379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cabee-0f9e-439a-9f6a-a23ad34b547e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.065131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053b3f5-2a2d-4d4d-84e6-bda69fc7142b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.067847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053b3f5-2a2d-4d4d-84e6-bda69fc7142b admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.068150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053b3f5-2a2d-4d4d-84e6-bda69fc7142b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.068866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053b3f5-2a2d-4d4d-84e6-bda69fc7142b admin admin] PATH_INFO: `/v3/projects/8ba929f19a2c479ab6bbf8db1a064a01/users/0813acc01d2f44d480a4e4c1d570db54/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.091817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053b3f5-2a2d-4d4d-84e6-bda69fc7142b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba929f19a2c479ab6bbf8db1a064a01, user_id=0813acc01d2f44d480a4e4c1d570db54, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.093842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 797/1598] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:17.094898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa72e3-2e18-43f0-919b-779ea99abd5a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.095219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053b3f5-2a2d-4d4d-84e6-bda69fc7142b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.108006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa72e3-2e18-43f0-919b-779ea99abd5a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.115997 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 802/1599] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:17 2022] PUT /identity/v3/projects/8ba929f19a2c479ab6bbf8db1a064a01/users/0813acc01d2f44d480a4e4c1d570db54/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:17.117233 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053b3f5-2a2d-4d4d-84e6-bda69fc7142b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.136552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:17.184905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b5c05-f534-4193-be8c-dc3a8ce8ca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.185729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b5c05-f534-4193-be8c-dc3a8ce8ca8b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.186062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b5c05-f534-4193-be8c-dc3a8ce8ca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.186332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b5c05-f534-4193-be8c-dc3a8ce8ca8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.187095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b5c05-f534-4193-be8c-dc3a8ce8ca8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.188813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b5c05-f534-4193-be8c-dc3a8ce8ca8b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.202664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 798/1600] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:17.203788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b5c05-f534-4193-be8c-dc3a8ce8ca8b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.212374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053b3f5-2a2d-4d4d-84e6-bda69fc7142b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.216980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b5c05-f534-4193-be8c-dc3a8ce8ca8b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.218725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7d5e5-24ce-4060-abb3-20008e54c948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.219928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7d5e5-24ce-4060-abb3-20008e54c948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.220595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7d5e5-24ce-4060-abb3-20008e54c948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.221217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7d5e5-24ce-4060-abb3-20008e54c948 admin admin] PATH_INFO: `/v3/projects/367a52be93e24b5eaf528a48342d47b3/users/0965444614ae486e80e111f4126b52f1/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.224232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420a317-7c0d-4177-881c-0e2352ae0dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.225368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420a317-7c0d-4177-881c-0e2352ae0dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.225647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420a317-7c0d-4177-881c-0e2352ae0dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.225990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420a317-7c0d-4177-881c-0e2352ae0dd1 admin admin] PATH_INFO: `/v3/projects/dab2ba718dad487fa0f174b8b0964286/users/b5871ced3beb4decacc3a20afe523349/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.244435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420a317-7c0d-4177-881c-0e2352ae0dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dab2ba718dad487fa0f174b8b0964286, user_id=b5871ced3beb4decacc3a20afe523349, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.245124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7d5e5-24ce-4060-abb3-20008e54c948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=367a52be93e24b5eaf528a48342d47b3, user_id=0965444614ae486e80e111f4126b52f1, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.247201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420a317-7c0d-4177-881c-0e2352ae0dd1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.248316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7d5e5-24ce-4060-abb3-20008e54c948 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.270250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 799/1601] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:17 2022] PUT /identity/v3/projects/dab2ba718dad487fa0f174b8b0964286/users/b5871ced3beb4decacc3a20afe523349/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:17.270479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 803/1602] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:17 2022] PUT /identity/v3/projects/367a52be93e24b5eaf528a48342d47b3/users/0965444614ae486e80e111f4126b52f1/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:17.270583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420a317-7c0d-4177-881c-0e2352ae0dd1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.270675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7d5e5-24ce-4060-abb3-20008e54c948 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.350749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420a317-7c0d-4177-881c-0e2352ae0dd1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.355255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7d5e5-24ce-4060-abb3-20008e54c948 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.355819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bb37b-aaa1-4ad2-8b55-28640cb89089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.357082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb37b-aaa1-4ad2-8b55-28640cb89089 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.357366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb37b-aaa1-4ad2-8b55-28640cb89089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.357636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb37b-aaa1-4ad2-8b55-28640cb89089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.358378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bb37b-aaa1-4ad2-8b55-28640cb89089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.360568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70136ade-f3ef-4078-9ac9-7e1f70941811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.360708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bb37b-aaa1-4ad2-8b55-28640cb89089 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.361657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70136ade-f3ef-4078-9ac9-7e1f70941811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.362023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70136ade-f3ef-4078-9ac9-7e1f70941811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.362386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70136ade-f3ef-4078-9ac9-7e1f70941811 admin admin] PATH_INFO: `/v3/projects/6001641b8786432fbe8c94c64855139a/users/0b03a4195935439e923932201f357623/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.375844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 800/1603] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:17.377485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bb37b-aaa1-4ad2-8b55-28640cb89089 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.382075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70136ade-f3ef-4078-9ac9-7e1f70941811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6001641b8786432fbe8c94c64855139a, user_id=0b03a4195935439e923932201f357623, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.385530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70136ade-f3ef-4078-9ac9-7e1f70941811 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.391062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bb37b-aaa1-4ad2-8b55-28640cb89089 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.397094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3228-c092-478a-8473-88b9f29f2b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.397672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3228-c092-478a-8473-88b9f29f2b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.398133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3228-c092-478a-8473-88b9f29f2b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.398412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3228-c092-478a-8473-88b9f29f2b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.399187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b3228-c092-478a-8473-88b9f29f2b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.401285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b3228-c092-478a-8473-88b9f29f2b13 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.410482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 804/1604] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:17 2022] PUT /identity/v3/projects/6001641b8786432fbe8c94c64855139a/users/0b03a4195935439e923932201f357623/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:17.412448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b8ee0-4048-47b6-8534-99b089510ab6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.412448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b8ee0-4048-47b6-8534-99b089510ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.412448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b8ee0-4048-47b6-8534-99b089510ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.444599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9f1b8ee0-4048-47b6-8534-99b089510ab6 None None] MFA Rules not processed for user `0813acc01d2f44d480a4e4c1d570db54`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:17.446071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9f1b8ee0-4048-47b6-8534-99b089510ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:17.447333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:17.455529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 801/1605] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:17 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:17.457155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3228-c092-478a-8473-88b9f29f2b13 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.529022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 805/1606] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:17.530436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6544688-e941-45a3-afb8-3a668d5f8339 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.530720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6544688-e941-45a3-afb8-3a668d5f8339 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.530991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6544688-e941-45a3-afb8-3a668d5f8339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.542607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3228-c092-478a-8473-88b9f29f2b13 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.543990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:17.550410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e38f9-c98d-4ea1-91dd-c17cfc704e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.551980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e38f9-c98d-4ea1-91dd-c17cfc704e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.552461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e38f9-c98d-4ea1-91dd-c17cfc704e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.552878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e38f9-c98d-4ea1-91dd-c17cfc704e26 admin admin] PATH_INFO: `/v3/projects/625074600ce14680b0f41382f409a450/users/907c2ffd4b5e4292a4fae1bf11e53475/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.560793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e6544688-e941-45a3-afb8-3a668d5f8339 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:17.562445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e6544688-e941-45a3-afb8-3a668d5f8339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:17.572453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e38f9-c98d-4ea1-91dd-c17cfc704e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625074600ce14680b0f41382f409a450, user_id=907c2ffd4b5e4292a4fae1bf11e53475, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.574970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e38f9-c98d-4ea1-91dd-c17cfc704e26 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.577996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 806/1607] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:17.581334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13d22a-dc2c-4136-85c0-a15f34e5d4f1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.581334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13d22a-dc2c-4136-85c0-a15f34e5d4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.581334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13d22a-dc2c-4136-85c0-a15f34e5d4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.609472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 802/1608] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:17 2022] PUT /identity/v3/projects/625074600ce14680b0f41382f409a450/users/907c2ffd4b5e4292a4fae1bf11e53475/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:17.609472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e38f9-c98d-4ea1-91dd-c17cfc704e26 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.631351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cf13d22a-dc2c-4136-85c0-a15f34e5d4f1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:17.632622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cf13d22a-dc2c-4136-85c0-a15f34e5d4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:17.672330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e38f9-c98d-4ea1-91dd-c17cfc704e26 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.682468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604575db-829b-414e-a707-d780b8fa333a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.684975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604575db-829b-414e-a707-d780b8fa333a admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.685307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604575db-829b-414e-a707-d780b8fa333a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.685605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604575db-829b-414e-a707-d780b8fa333a admin admin] PATH_INFO: `/v3/projects/570d2b4d5c384612ba17efc8b5bd02ec/users/77cf43c378a9413dbd799a00bfa21eb8/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.708973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604575db-829b-414e-a707-d780b8fa333a admin admin] RBAC: Authorizing `identity:create_grant(project_id=570d2b4d5c384612ba17efc8b5bd02ec, user_id=77cf43c378a9413dbd799a00bfa21eb8, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.712202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604575db-829b-414e-a707-d780b8fa333a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.722217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 807/1609] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:17.722217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146f6de-fa49-4cda-a991-d648dced4312 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.722217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146f6de-fa49-4cda-a991-d648dced4312 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.722217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146f6de-fa49-4cda-a991-d648dced4312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.733392 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 803/1610] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:17 2022] PUT /identity/v3/projects/570d2b4d5c384612ba17efc8b5bd02ec/users/77cf43c378a9413dbd799a00bfa21eb8/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:17.734635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604575db-829b-414e-a707-d780b8fa333a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.760184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f146f6de-fa49-4cda-a991-d648dced4312 None None] MFA Rules not processed for user `0b03a4195935439e923932201f357623`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:17.761388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f146f6de-fa49-4cda-a991-d648dced4312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:17.831739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604575db-829b-414e-a707-d780b8fa333a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.839329 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec91557-e488-4092-8cc2-ed454302001d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.840240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec91557-e488-4092-8cc2-ed454302001d admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.840698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec91557-e488-4092-8cc2-ed454302001d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.841074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec91557-e488-4092-8cc2-ed454302001d admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.841863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec91557-e488-4092-8cc2-ed454302001d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.843925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec91557-e488-4092-8cc2-ed454302001d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.856093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:17.859128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 808/1611] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:17.860940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8543f-822e-4fc1-b2c1-51f8bf6ac736 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.860940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8543f-822e-4fc1-b2c1-51f8bf6ac736 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.860940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8543f-822e-4fc1-b2c1-51f8bf6ac736 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.862653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 809/1612] 10.222.0.20 () {54 vars in 961 bytes} [Mon Jul 11 20:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:57:17.873126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8543f-822e-4fc1-b2c1-51f8bf6ac736 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:17.878393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b2b8543f-822e-4fc1-b2c1-51f8bf6ac736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:17.884543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 804/1613] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:17 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:17.886379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157be97-b92d-4264-8649-645982e76f90 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.886782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157be97-b92d-4264-8649-645982e76f90 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.888782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157be97-b92d-4264-8649-645982e76f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.945349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c157be97-b92d-4264-8649-645982e76f90 None None] MFA Rules not processed for user `907c2ffd4b5e4292a4fae1bf11e53475`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:17.947259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c157be97-b92d-4264-8649-645982e76f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:17.962788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8543f-822e-4fc1-b2c1-51f8bf6ac736 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:17.967063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87ed7b-b58a-4d28-ac06-801108030290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:17.967618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87ed7b-b58a-4d28-ac06-801108030290 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:17.968008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87ed7b-b58a-4d28-ac06-801108030290 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:17.968416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87ed7b-b58a-4d28-ac06-801108030290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:17.982136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87ed7b-b58a-4d28-ac06-801108030290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:17.983976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87ed7b-b58a-4d28-ac06-801108030290 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:17.995502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 810/1614] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:17.996509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87ed7b-b58a-4d28-ac06-801108030290 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.008271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87ed7b-b58a-4d28-ac06-801108030290 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.012521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d62220-def5-450d-8924-ab3a5e43b5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.013356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d62220-def5-450d-8924-ab3a5e43b5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.013701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d62220-def5-450d-8924-ab3a5e43b5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.014057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d62220-def5-450d-8924-ab3a5e43b5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.025778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d62220-def5-450d-8924-ab3a5e43b5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.027093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 805/1615] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:18.028230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d62220-def5-450d-8924-ab3a5e43b5a8 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.028898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013569a-c7c8-4291-b8cb-d1a17816de25 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.029254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013569a-c7c8-4291-b8cb-d1a17816de25 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.029598 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013569a-c7c8-4291-b8cb-d1a17816de25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.039141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 811/1616] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:18.040656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d7368-843d-4411-85f0-6eb6afe3c6b9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.041014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d7368-843d-4411-85f0-6eb6afe3c6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.041356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d7368-843d-4411-85f0-6eb6afe3c6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.049020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e013569a-c7c8-4291-b8cb-d1a17816de25 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:18.050339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e013569a-c7c8-4291-b8cb-d1a17816de25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:18.066451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 806/1617] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:18.074927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013569a-c7c8-4291-b8cb-d1a17816de25 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.090201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013569a-c7c8-4291-b8cb-d1a17816de25 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.095799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9d292-6e6f-4e0e-8c77-2c84b4d560fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.098299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9d292-6e6f-4e0e-8c77-2c84b4d560fd admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.098299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9d292-6e6f-4e0e-8c77-2c84b4d560fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.098609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:18.172363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9d292-6e6f-4e0e-8c77-2c84b4d560fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.172363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9d292-6e6f-4e0e-8c77-2c84b4d560fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.172363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9d292-6e6f-4e0e-8c77-2c84b4d560fd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.174231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0d4d7368-843d-4411-85f0-6eb6afe3c6b9 None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:18.174231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0d4d7368-843d-4411-85f0-6eb6afe3c6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:18.175875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 807/1618] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:18.175875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7917268-5a88-49a3-99ad-09aa30495ae0 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.175875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7917268-5a88-49a3-99ad-09aa30495ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.175875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7917268-5a88-49a3-99ad-09aa30495ae0 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.175875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 808/1619] 10.222.0.20 () {54 vars in 961 bytes} [Mon Jul 11 20:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:57:18.175875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7917268-5a88-49a3-99ad-09aa30495ae0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.181711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7917268-5a88-49a3-99ad-09aa30495ae0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.201222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c93962-60a7-4dca-9280-052b556f1c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.201925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c93962-60a7-4dca-9280-052b556f1c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.202216 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c93962-60a7-4dca-9280-052b556f1c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.202498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c93962-60a7-4dca-9280-052b556f1c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.203497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c93962-60a7-4dca-9280-052b556f1c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.206542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c93962-60a7-4dca-9280-052b556f1c58 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.255913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 809/1620] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:18 2022] POST /identity/v3/projects => generated 355 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:18.257380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c93962-60a7-4dca-9280-052b556f1c58 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.273494 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 812/1621] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Jul 11 20:57:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:18.274819 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d7368-843d-4411-85f0-6eb6afe3c6b9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.294384 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d7368-843d-4411-85f0-6eb6afe3c6b9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.351850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c93962-60a7-4dca-9280-052b556f1c58 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.356827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb4465-7f90-4e3e-9ad7-7c3b85489f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.357893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb4465-7f90-4e3e-9ad7-7c3b85489f2a service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.358323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb4465-7f90-4e3e-9ad7-7c3b85489f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.359339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb4465-7f90-4e3e-9ad7-7c3b85489f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.384301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c1900-78b9-4336-9e2f-b70563df2200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.385680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c1900-78b9-4336-9e2f-b70563df2200 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.385971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c1900-78b9-4336-9e2f-b70563df2200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.386249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c1900-78b9-4336-9e2f-b70563df2200 admin admin] PATH_INFO: `/v3/projects/cab69e10991246f48b060241ea7006f0/users/f4ca8304f1c94a309e087b4dab970f86/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.407269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c1900-78b9-4336-9e2f-b70563df2200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab69e10991246f48b060241ea7006f0, user_id=f4ca8304f1c94a309e087b4dab970f86, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.411866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c1900-78b9-4336-9e2f-b70563df2200 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.431617 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 813/1622] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:18 2022] PUT /identity/v3/projects/cab69e10991246f48b060241ea7006f0/users/f4ca8304f1c94a309e087b4dab970f86/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:18.433322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d4d4c-67a4-4f4c-a009-15413bf50ec7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.433607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d4d4c-67a4-4f4c-a009-15413bf50ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.433890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d4d4c-67a4-4f4c-a009-15413bf50ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.448363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:18.456866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb4465-7f90-4e3e-9ad7-7c3b85489f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.461229 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb4465-7f90-4e3e-9ad7-7c3b85489f2a service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.461561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-277d4d4c-67a4-4f4c-a009-15413bf50ec7 None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:18.466489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-277d4d4c-67a4-4f4c-a009-15413bf50ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:18.482833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 810/1623] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:18.483721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb4465-7f90-4e3e-9ad7-7c3b85489f2a service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.500227 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb4465-7f90-4e3e-9ad7-7c3b85489f2a service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.577375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 814/1624] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Jul 11 20:57:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:18.579136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d4d4c-67a4-4f4c-a009-15413bf50ec7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.584651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303051ee-046a-4587-bfb1-c5e8028b2b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.585354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303051ee-046a-4587-bfb1-c5e8028b2b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.585636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303051ee-046a-4587-bfb1-c5e8028b2b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.585913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303051ee-046a-4587-bfb1-c5e8028b2b06 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.586621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303051ee-046a-4587-bfb1-c5e8028b2b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.589236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303051ee-046a-4587-bfb1-c5e8028b2b06 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.597524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d4d4c-67a4-4f4c-a009-15413bf50ec7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.606287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b669d86-d44a-4d1d-9fa1-35e782194c4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.607058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b669d86-d44a-4d1d-9fa1-35e782194c4d service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.607835 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b669d86-d44a-4d1d-9fa1-35e782194c4d service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.607835 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b669d86-d44a-4d1d-9fa1-35e782194c4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.626578 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:18.629735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 811/1625] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:18 2022] POST /identity/v3/users => generated 284 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:18.630648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303051ee-046a-4587-bfb1-c5e8028b2b06 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.649968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303051ee-046a-4587-bfb1-c5e8028b2b06 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.655798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1c3b0-221e-4995-909c-ad1bca3be455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.656522 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1c3b0-221e-4995-909c-ad1bca3be455 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.656803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1c3b0-221e-4995-909c-ad1bca3be455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.657076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1c3b0-221e-4995-909c-ad1bca3be455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.657821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1c3b0-221e-4995-909c-ad1bca3be455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.659439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1c3b0-221e-4995-909c-ad1bca3be455 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.689965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 812/1626] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:18.690943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1c3b0-221e-4995-909c-ad1bca3be455 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.725808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1c3b0-221e-4995-909c-ad1bca3be455 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.733633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f34e6-8478-4a11-a82e-dc9ffd15c805 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.734675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f34e6-8478-4a11-a82e-dc9ffd15c805 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.735458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f34e6-8478-4a11-a82e-dc9ffd15c805 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.735744 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f34e6-8478-4a11-a82e-dc9ffd15c805 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.737571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b669d86-d44a-4d1d-9fa1-35e782194c4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.741210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b669d86-d44a-4d1d-9fa1-35e782194c4d service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.769507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 815/1627] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:18.771918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b669d86-d44a-4d1d-9fa1-35e782194c4d service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.797990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b669d86-d44a-4d1d-9fa1-35e782194c4d service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.807427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19d6b0-fb30-498b-b594-460f6e19e487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.808796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d6b0-fb30-498b-b594-460f6e19e487 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.809151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d6b0-fb30-498b-b594-460f6e19e487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.809519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d6b0-fb30-498b-b594-460f6e19e487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.810406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19d6b0-fb30-498b-b594-460f6e19e487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.813507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19d6b0-fb30-498b-b594-460f6e19e487 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.840101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 816/1628] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:18.861700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19d6b0-fb30-498b-b594-460f6e19e487 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.879060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19d6b0-fb30-498b-b594-460f6e19e487 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:18.886065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63c4fd-dc7d-46c0-adde-6b1fdf76518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:18.887001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63c4fd-dc7d-46c0-adde-6b1fdf76518d admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.887751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63c4fd-dc7d-46c0-adde-6b1fdf76518d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.888149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63c4fd-dc7d-46c0-adde-6b1fdf76518d admin admin] PATH_INFO: `/v3/projects/cab69e10991246f48b060241ea7006f0/users/f4ca8304f1c94a309e087b4dab970f86/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:18.892560 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63c4fd-dc7d-46c0-adde-6b1fdf76518d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab69e10991246f48b060241ea7006f0, user_id=f4ca8304f1c94a309e087b4dab970f86, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.896531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63c4fd-dc7d-46c0-adde-6b1fdf76518d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.916061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f34e6-8478-4a11-a82e-dc9ffd15c805 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:18.917768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f34e6-8478-4a11-a82e-dc9ffd15c805 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:18.923157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 817/1629] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:18 2022] PUT /identity/v3/projects/cab69e10991246f48b060241ea7006f0/users/f4ca8304f1c94a309e087b4dab970f86/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:18.924030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63c4fd-dc7d-46c0-adde-6b1fdf76518d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:18.959811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 813/1630] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:18.961233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8712ff-b116-40cc-9bb2-3127002b5ae6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:18.961521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8712ff-b116-40cc-9bb2-3127002b5ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:18.961796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8712ff-b116-40cc-9bb2-3127002b5ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.022117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-dd8712ff-b116-40cc-9bb2-3127002b5ae6 None None] MFA Rules not processed for user `f4ca8304f1c94a309e087b4dab970f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:19.023365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dd8712ff-b116-40cc-9bb2-3127002b5ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:19.073855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63c4fd-dc7d-46c0-adde-6b1fdf76518d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:19.081146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92ff95-0241-483c-a23a-ad90cf70276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:19.083056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92ff95-0241-483c-a23a-ad90cf70276e admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.083975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92ff95-0241-483c-a23a-ad90cf70276e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.084460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92ff95-0241-483c-a23a-ad90cf70276e admin admin] PATH_INFO: `/v3/projects/64b8e670b8c2436fa7e70a22be497af6/users/e5f41154e61f47e8ba7a9830b18f827f/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.106608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92ff95-0241-483c-a23a-ad90cf70276e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b8e670b8c2436fa7e70a22be497af6, user_id=e5f41154e61f47e8ba7a9830b18f827f, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:19.110665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92ff95-0241-483c-a23a-ad90cf70276e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:19.148806 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 818/1631] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:18 2022] PUT /identity/v3/projects/64b8e670b8c2436fa7e70a22be497af6/users/e5f41154e61f47e8ba7a9830b18f827f/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:19.152595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92ff95-0241-483c-a23a-ad90cf70276e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:19.166596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:19.186478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 814/1632] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:19.196689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8712ff-b116-40cc-9bb2-3127002b5ae6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:19.352021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8712ff-b116-40cc-9bb2-3127002b5ae6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:19.354853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92ff95-0241-483c-a23a-ad90cf70276e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:19.366117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20da70a-6efa-4c01-b9bd-337ea1e7ff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:19.367087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da70a-6efa-4c01-b9bd-337ea1e7ff21 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.367087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da70a-6efa-4c01-b9bd-337ea1e7ff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.367087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da70a-6efa-4c01-b9bd-337ea1e7ff21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.367665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20da70a-6efa-4c01-b9bd-337ea1e7ff21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:19.369566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20da70a-6efa-4c01-b9bd-337ea1e7ff21 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:19.402895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 819/1633] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:19.408907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20da70a-6efa-4c01-b9bd-337ea1e7ff21 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:19.430504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20da70a-6efa-4c01-b9bd-337ea1e7ff21 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:19.441352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160e6a8-d630-49c1-9c3e-88655dfb8f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:19.447550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160e6a8-d630-49c1-9c3e-88655dfb8f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.447824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160e6a8-d630-49c1-9c3e-88655dfb8f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.448193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160e6a8-d630-49c1-9c3e-88655dfb8f34 admin admin] PATH_INFO: `/v3/projects/64b8e670b8c2436fa7e70a22be497af6/users/e5f41154e61f47e8ba7a9830b18f827f/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.452890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160e6a8-d630-49c1-9c3e-88655dfb8f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b8e670b8c2436fa7e70a22be497af6, user_id=e5f41154e61f47e8ba7a9830b18f827f, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:19.462949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160e6a8-d630-49c1-9c3e-88655dfb8f34 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:19.467846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:19.477963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c45b0-11d0-4cc9-aee8-1e405ec58562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:19.479632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c45b0-11d0-4cc9-aee8-1e405ec58562 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.481440 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c45b0-11d0-4cc9-aee8-1e405ec58562 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.481844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c45b0-11d0-4cc9-aee8-1e405ec58562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.486180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 820/1634] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:19 2022] PUT /identity/v3/projects/64b8e670b8c2436fa7e70a22be497af6/users/e5f41154e61f47e8ba7a9830b18f827f/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:19.488005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eeb64-2714-4f2b-a05e-f6451096f270 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.488354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eeb64-2714-4f2b-a05e-f6451096f270 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.488634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eeb64-2714-4f2b-a05e-f6451096f270 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.490204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 821/1635] 10.222.0.20 () {54 vars in 912 bytes} [Mon Jul 11 20:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:57:19.491081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d179bf-310f-4ed6-a203-1bfed24cdca9 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.491422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d179bf-310f-4ed6-a203-1bfed24cdca9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.491701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d179bf-310f-4ed6-a203-1bfed24cdca9 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.497336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 822/1636] 10.222.0.20 () {54 vars in 912 bytes} [Mon Jul 11 20:57:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:57:19.499128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2b28-7d8b-4571-948a-b0b67dc2b404 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.499423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2b28-7d8b-4571-948a-b0b67dc2b404 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.499788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2b28-7d8b-4571-948a-b0b67dc2b404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.521116 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-128e2b28-7d8b-4571-948a-b0b67dc2b404 None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:19.522360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-128e2b28-7d8b-4571-948a-b0b67dc2b404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:19.622340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c45b0-11d0-4cc9-aee8-1e405ec58562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:19.623829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c45b0-11d0-4cc9-aee8-1e405ec58562 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:19.646308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 815/1637] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:19.647624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693630f-0b5f-4cba-b8ed-ccaeef337e82 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.647911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693630f-0b5f-4cba-b8ed-ccaeef337e82 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.648247 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693630f-0b5f-4cba-b8ed-ccaeef337e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.651246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 823/1638] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 20:57:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:19.652952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952a517-e67e-4785-83b8-cc55b96c0f63 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.653236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952a517-e67e-4785-83b8-cc55b96c0f63 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.653511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952a517-e67e-4785-83b8-cc55b96c0f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.690494 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7952a517-e67e-4785-83b8-cc55b96c0f63 None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:19.695872 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7952a517-e67e-4785-83b8-cc55b96c0f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:19.711444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c693630f-0b5f-4cba-b8ed-ccaeef337e82 None None] MFA Rules not processed for user `e5f41154e61f47e8ba7a9830b18f827f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:19.713153 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c693630f-0b5f-4cba-b8ed-ccaeef337e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:19.721400 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 824/1639] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 20:57:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:19.722742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952a517-e67e-4785-83b8-cc55b96c0f63 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:19.830716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:19.856546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952a517-e67e-4785-83b8-cc55b96c0f63 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:19.864763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa9868-c4be-46a7-974b-5ec8c513c8ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:19.865437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa9868-c4be-46a7-974b-5ec8c513c8ea service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.865713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa9868-c4be-46a7-974b-5ec8c513c8ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.865986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa9868-c4be-46a7-974b-5ec8c513c8ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.876370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 816/1640] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:19 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:19.877775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693630f-0b5f-4cba-b8ed-ccaeef337e82 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:19.892340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa9868-c4be-46a7-974b-5ec8c513c8ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:19.893778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa9868-c4be-46a7-974b-5ec8c513c8ea service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:19.903655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693630f-0b5f-4cba-b8ed-ccaeef337e82 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:19.913968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f6e0b-1be0-47fa-ae21-dc0b4057df84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:19.917016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f6e0b-1be0-47fa-ae21-dc0b4057df84 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:19.917256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 825/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:19.917391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f6e0b-1be0-47fa-ae21-dc0b4057df84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:19.917700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f6e0b-1be0-47fa-ae21-dc0b4057df84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:19.950083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f6e0b-1be0-47fa-ae21-dc0b4057df84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:19.952485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f6e0b-1be0-47fa-ae21-dc0b4057df84 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:19.967572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 817/1642] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:20.092535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa9868-c4be-46a7-974b-5ec8c513c8ea service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:20.100247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa9868-c4be-46a7-974b-5ec8c513c8ea service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:20.106638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607291f-7f5a-4e2e-a06e-b3b7a18d7ee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:20.107355 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607291f-7f5a-4e2e-a06e-b3b7a18d7ee7 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:20.107637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607291f-7f5a-4e2e-a06e-b3b7a18d7ee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:20.107908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607291f-7f5a-4e2e-a06e-b3b7a18d7ee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:20.263118 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607291f-7f5a-4e2e-a06e-b3b7a18d7ee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:20.266974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607291f-7f5a-4e2e-a06e-b3b7a18d7ee7 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:20.304712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 826/1643] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:20.698328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f6e0b-1be0-47fa-ae21-dc0b4057df84 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:20.710540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f6e0b-1be0-47fa-ae21-dc0b4057df84 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:20.721235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ff976-f93f-42df-85e0-3ff91a84bd0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:20.722100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ff976-f93f-42df-85e0-3ff91a84bd0f service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:20.722618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ff976-f93f-42df-85e0-3ff91a84bd0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:20.723069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ff976-f93f-42df-85e0-3ff91a84bd0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:20.746771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ff976-f93f-42df-85e0-3ff91a84bd0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:20.748468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ff976-f93f-42df-85e0-3ff91a84bd0f service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:20.778417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 818/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:20 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:21.093672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607291f-7f5a-4e2e-a06e-b3b7a18d7ee7 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:21.133677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:21.211066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607291f-7f5a-4e2e-a06e-b3b7a18d7ee7 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:21.226440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:21.227730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:21.228264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:21.228733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] PATH_INFO: `/v3/users/0813acc01d2f44d480a4e4c1d570db54` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:21.230911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0813acc01d2f44d480a4e4c1d570db54)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:21.232215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:57:21.232215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 11 20:57:21.232215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: Jul 11 20:57:21.234116 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:21.373333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0813acc01d2f44d480a4e4c1d570db54'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:21.398710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0813acc01d2f44d480a4e4c1d570db54'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:21.399615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0813acc01d2f44d480a4e4c1d570db54'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:21.426160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0813acc01d2f44d480a4e4c1d570db54'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:21.463582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0813acc01d2f44d480a4e4c1d570db54'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:21.497842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 827/1645] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:57:21 2022] DELETE /identity/v3/users/0813acc01d2f44d480a4e4c1d570db54 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:21.499665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446847fe-18cd-4b69-9fa0-4830d412609f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:21.500069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446847fe-18cd-4b69-9fa0-4830d412609f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:21.500457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446847fe-18cd-4b69-9fa0-4830d412609f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:21.558961 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-446847fe-18cd-4b69-9fa0-4830d412609f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:21.565353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-446847fe-18cd-4b69-9fa0-4830d412609f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:21.697369 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:21.699032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:21.806147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 819/1646] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:21.833965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446847fe-18cd-4b69-9fa0-4830d412609f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:21.835709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-446847fe-18cd-4b69-9fa0-4830d412609f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:21.900521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38941f9f-bf2e-4744-9ea6-5db82ee71d8b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:21.922135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4756a04-0adf-477d-b706-5f24aee0f33b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:21.923040 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4756a04-0adf-477d-b706-5f24aee0f33b service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:21.923425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4756a04-0adf-477d-b706-5f24aee0f33b service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:21.923803 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4756a04-0adf-477d-b706-5f24aee0f33b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:21.929897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b4756a04-0adf-477d-b706-5f24aee0f33b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:22.035276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446847fe-18cd-4b69-9fa0-4830d412609f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:22.046434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:22.055839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4756a04-0adf-477d-b706-5f24aee0f33b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:22.059785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4756a04-0adf-477d-b706-5f24aee0f33b service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:22.063326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14927c39-61d1-41c0-b40c-95d83860c83a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:22.064200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927c39-61d1-41c0-b40c-95d83860c83a service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:22.064511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927c39-61d1-41c0-b40c-95d83860c83a service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:22.064856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927c39-61d1-41c0-b40c-95d83860c83a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:22.085358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 828/1647] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:21 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:22.085587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14927c39-61d1-41c0-b40c-95d83860c83a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:22.087812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14927c39-61d1-41c0-b40c-95d83860c83a service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:22.104300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 820/1648] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:57:23.981212 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4756a04-0adf-477d-b706-5f24aee0f33b service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:23.983996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b4756a04-0adf-477d-b706-5f24aee0f33b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:24.033945 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4756a04-0adf-477d-b706-5f24aee0f33b service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:24.052399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:24.056061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:24.057021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:24.058901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] PATH_INFO: `/v3/projects/8ba929f19a2c479ab6bbf8db1a064a01` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:24.064535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba929f19a2c479ab6bbf8db1a064a01)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:24.073115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:24.225688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba929f19a2c479ab6bbf8db1a064a01', 'request_id': 'req-11950a8c-5d86-4bbc-aedd-169fc8183e70'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:24.247524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba929f19a2c479ab6bbf8db1a064a01', 'request_id': 'req-11950a8c-5d86-4bbc-aedd-169fc8183e70'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:24.275331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba929f19a2c479ab6bbf8db1a064a01', 'request_id': 'req-11950a8c-5d86-4bbc-aedd-169fc8183e70'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:24.298638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] The token cache is being invalidate because project 8ba929f19a2c479ab6bbf8db1a064a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:57:24.299092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:24.302097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11950a8c-5d86-4bbc-aedd-169fc8183e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:24.308069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 829/1649] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:57:23 2022] DELETE /identity/v3/projects/8ba929f19a2c479ab6bbf8db1a064a01 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:24.313056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788f224-6e82-45ab-8059-3641f8b3b915 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:24.313655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788f224-6e82-45ab-8059-3641f8b3b915 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:24.313655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788f224-6e82-45ab-8059-3641f8b3b915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:24.347242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0788f224-6e82-45ab-8059-3641f8b3b915 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:24.348678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0788f224-6e82-45ab-8059-3641f8b3b915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:24.536753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 821/1650] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:24.545080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fcb7d-b685-41f5-a58c-504ee094d4c4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:24.546179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fcb7d-b685-41f5-a58c-504ee094d4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:24.546656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fcb7d-b685-41f5-a58c-504ee094d4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:24.589642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a08fcb7d-b685-41f5-a58c-504ee094d4c4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:24.591945 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a08fcb7d-b685-41f5-a58c-504ee094d4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:24.620547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788f224-6e82-45ab-8059-3641f8b3b915 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:24.625884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 830/1651] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:24.642301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788f224-6e82-45ab-8059-3641f8b3b915 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:24.649363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07135ba3-5de2-41e1-a036-e9f9b9b493fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:24.650396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07135ba3-5de2-41e1-a036-e9f9b9b493fb admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:24.651502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07135ba3-5de2-41e1-a036-e9f9b9b493fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:24.652608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07135ba3-5de2-41e1-a036-e9f9b9b493fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:24.653566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07135ba3-5de2-41e1-a036-e9f9b9b493fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:24.654128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07135ba3-5de2-41e1-a036-e9f9b9b493fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:24.656054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07135ba3-5de2-41e1-a036-e9f9b9b493fb admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:24.658279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:24.683092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 822/1652] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:24.687922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fcb7d-b685-41f5-a58c-504ee094d4c4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:24.700723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fcb7d-b685-41f5-a58c-504ee094d4c4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:24.713320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b652271-de48-4ebe-944c-9520ac092aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:24.714127 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b652271-de48-4ebe-944c-9520ac092aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:24.714693 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b652271-de48-4ebe-944c-9520ac092aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:24.715130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b652271-de48-4ebe-944c-9520ac092aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:24.716035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b652271-de48-4ebe-944c-9520ac092aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:24.718504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b652271-de48-4ebe-944c-9520ac092aaf admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:24.806478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07135ba3-5de2-41e1-a036-e9f9b9b493fb admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:24.808360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 831/1653] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:24 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:24.836687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07135ba3-5de2-41e1-a036-e9f9b9b493fb admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:24.957799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0906f-0cf9-4fae-a40e-2151b5fd7311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:24.958704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0906f-0cf9-4fae-a40e-2151b5fd7311 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:24.958822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0906f-0cf9-4fae-a40e-2151b5fd7311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:24.959030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0906f-0cf9-4fae-a40e-2151b5fd7311 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:24.959746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0906f-0cf9-4fae-a40e-2151b5fd7311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:24.961740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0906f-0cf9-4fae-a40e-2151b5fd7311 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:25.023887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 823/1654] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:24 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:57:25.029253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b652271-de48-4ebe-944c-9520ac092aaf admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:25.055909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b652271-de48-4ebe-944c-9520ac092aaf admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:25.108579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d2c97-b6ed-48e0-8d02-8c1a92c0fb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:25.109287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d2c97-b6ed-48e0-8d02-8c1a92c0fb79 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:25.109578 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d2c97-b6ed-48e0-8d02-8c1a92c0fb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:25.115489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d2c97-b6ed-48e0-8d02-8c1a92c0fb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:25.118519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d2c97-b6ed-48e0-8d02-8c1a92c0fb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:25.123381 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d2c97-b6ed-48e0-8d02-8c1a92c0fb79 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:25.171935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 832/1655] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:25.174950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0906f-0cf9-4fae-a40e-2151b5fd7311 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:25.198245 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0906f-0cf9-4fae-a40e-2151b5fd7311 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:25.220576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def264d3-d8bc-449c-9c59-ef0c7546f026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:25.221613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def264d3-d8bc-449c-9c59-ef0c7546f026 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:25.222159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def264d3-d8bc-449c-9c59-ef0c7546f026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:25.222499 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def264d3-d8bc-449c-9c59-ef0c7546f026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:25.232938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def264d3-d8bc-449c-9c59-ef0c7546f026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:25.236263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def264d3-d8bc-449c-9c59-ef0c7546f026 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:25.284773 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 824/1656] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:25.289165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d2c97-b6ed-48e0-8d02-8c1a92c0fb79 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:25.302918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d2c97-b6ed-48e0-8d02-8c1a92c0fb79 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:25.324872 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f17950-3a2a-4ca4-b6aa-20af8bc13410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:25.325840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f17950-3a2a-4ca4-b6aa-20af8bc13410 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:25.326057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f17950-3a2a-4ca4-b6aa-20af8bc13410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:25.326330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f17950-3a2a-4ca4-b6aa-20af8bc13410 admin admin] PATH_INFO: `/v3/projects/f93db91e0926470da7fc7446540e98e6/users/cca4ef98735c4c84bbf61b9371ee33a4/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:25.361044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f17950-3a2a-4ca4-b6aa-20af8bc13410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93db91e0926470da7fc7446540e98e6, user_id=cca4ef98735c4c84bbf61b9371ee33a4, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:25.363771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f17950-3a2a-4ca4-b6aa-20af8bc13410 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:25.397621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 833/1657] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:25 2022] PUT /identity/v3/projects/f93db91e0926470da7fc7446540e98e6/users/cca4ef98735c4c84bbf61b9371ee33a4/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:25.408349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b1e5e-5948-4ad1-973f-fcf089d5b880 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:25.408985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b1e5e-5948-4ad1-973f-fcf089d5b880 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:25.408985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b1e5e-5948-4ad1-973f-fcf089d5b880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:25.502907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e06b1e5e-5948-4ad1-973f-fcf089d5b880 None None] MFA Rules not processed for user `cca4ef98735c4c84bbf61b9371ee33a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:25.505026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e06b1e5e-5948-4ad1-973f-fcf089d5b880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:25.824700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 825/1658] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:25.829273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f17950-3a2a-4ca4-b6aa-20af8bc13410 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:26.000696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f17950-3a2a-4ca4-b6aa-20af8bc13410 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:26.027101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70565f-7d89-406c-96b1-ac3906b82ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:26.027811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70565f-7d89-406c-96b1-ac3906b82ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:26.028158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70565f-7d89-406c-96b1-ac3906b82ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:26.030503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70565f-7d89-406c-96b1-ac3906b82ef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:26.032804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70565f-7d89-406c-96b1-ac3906b82ef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:26.037347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70565f-7d89-406c-96b1-ac3906b82ef5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:26.104846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:26.116802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 834/1659] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:25 2022] POST /identity/v3/projects => generated 381 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:26.121204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b1e5e-5948-4ad1-973f-fcf089d5b880 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:26.166428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b1e5e-5948-4ad1-973f-fcf089d5b880 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:26.295571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff39b6-cc34-49b2-9f2b-e790d42a637f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:26.296198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff39b6-cc34-49b2-9f2b-e790d42a637f admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:26.296936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff39b6-cc34-49b2-9f2b-e790d42a637f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:26.297235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff39b6-cc34-49b2-9f2b-e790d42a637f admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:26.298131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff39b6-cc34-49b2-9f2b-e790d42a637f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:26.300826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff39b6-cc34-49b2-9f2b-e790d42a637f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:26.319783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:26.348346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 826/1660] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:26 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:26.352501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70565f-7d89-406c-96b1-ac3906b82ef5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:26.366451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70565f-7d89-406c-96b1-ac3906b82ef5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:26.389035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26a0c8-5f75-43dd-9490-557234bcc8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:26.389791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26a0c8-5f75-43dd-9490-557234bcc8ae admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:26.390049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26a0c8-5f75-43dd-9490-557234bcc8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:26.390329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26a0c8-5f75-43dd-9490-557234bcc8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:26.391092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26a0c8-5f75-43dd-9490-557234bcc8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:26.392863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26a0c8-5f75-43dd-9490-557234bcc8ae admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:26.418297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 835/1661] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:26.422746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff39b6-cc34-49b2-9f2b-e790d42a637f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:26.442967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff39b6-cc34-49b2-9f2b-e790d42a637f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:26.448431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51e895-7415-430f-8c36-db5d04cb36b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:26.449662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51e895-7415-430f-8c36-db5d04cb36b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:26.450157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51e895-7415-430f-8c36-db5d04cb36b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:26.450643 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51e895-7415-430f-8c36-db5d04cb36b8 admin admin] PATH_INFO: `/v3/projects/24987b91a2824f08ab36c532c3add1d1/users/35cbf3fa31ef4b50a794b28752da6e56/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:26.469364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51e895-7415-430f-8c36-db5d04cb36b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24987b91a2824f08ab36c532c3add1d1, user_id=35cbf3fa31ef4b50a794b28752da6e56, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:26.472469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51e895-7415-430f-8c36-db5d04cb36b8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:26.501758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 827/1662] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:26 2022] PUT /identity/v3/projects/24987b91a2824f08ab36c532c3add1d1/users/35cbf3fa31ef4b50a794b28752da6e56/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:26.504041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26a0c8-5f75-43dd-9490-557234bcc8ae admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:26.518738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331291ce-a183-4ca9-bcbd-39c3fe573630 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:26.519030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331291ce-a183-4ca9-bcbd-39c3fe573630 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:26.519344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331291ce-a183-4ca9-bcbd-39c3fe573630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:26.523273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26a0c8-5f75-43dd-9490-557234bcc8ae admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:26.578827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-331291ce-a183-4ca9-bcbd-39c3fe573630 None None] MFA Rules not processed for user `aa7e6c5842a84e91ac94c1107ad29158`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:26.580060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-331291ce-a183-4ca9-bcbd-39c3fe573630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:26.669820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecc2d2-abc4-44bb-93f3-aeeedc32db0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:26.670550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc2d2-abc4-44bb-93f3-aeeedc32db0c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:26.670836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc2d2-abc4-44bb-93f3-aeeedc32db0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:26.671108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc2d2-abc4-44bb-93f3-aeeedc32db0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:26.671861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecc2d2-abc4-44bb-93f3-aeeedc32db0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:26.673520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecc2d2-abc4-44bb-93f3-aeeedc32db0c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:26.718962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 836/1663] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:57:26.719931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecc2d2-abc4-44bb-93f3-aeeedc32db0c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:26.750215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecc2d2-abc4-44bb-93f3-aeeedc32db0c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:26.764347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100cb2f-4f2c-4dcb-ba3f-8e41aefd1d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:26.765185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100cb2f-4f2c-4dcb-ba3f-8e41aefd1d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:26.765631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100cb2f-4f2c-4dcb-ba3f-8e41aefd1d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:26.766048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100cb2f-4f2c-4dcb-ba3f-8e41aefd1d58 admin admin] PATH_INFO: `/v3/projects/24987b91a2824f08ab36c532c3add1d1/users/35cbf3fa31ef4b50a794b28752da6e56/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:26.769815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100cb2f-4f2c-4dcb-ba3f-8e41aefd1d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24987b91a2824f08ab36c532c3add1d1, user_id=35cbf3fa31ef4b50a794b28752da6e56, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:26.771412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 828/1664] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:26.774808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100cb2f-4f2c-4dcb-ba3f-8e41aefd1d58 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:26.781755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc918cb2-497d-4daa-b455-08d077a75634 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:26.782268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc918cb2-497d-4daa-b455-08d077a75634 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:26.782697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc918cb2-497d-4daa-b455-08d077a75634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:26.823781 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 837/1665] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:26 2022] PUT /identity/v3/projects/24987b91a2824f08ab36c532c3add1d1/users/35cbf3fa31ef4b50a794b28752da6e56/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:26.836078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5962c-5558-405a-9cb1-5dc503d57016 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:26.836488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5962c-5558-405a-9cb1-5dc503d57016 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:26.836862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5962c-5558-405a-9cb1-5dc503d57016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:26.845594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bc918cb2-497d-4daa-b455-08d077a75634 None None] MFA Rules not processed for user `aa7e6c5842a84e91ac94c1107ad29158`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:26.850479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bc918cb2-497d-4daa-b455-08d077a75634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:26.912828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e8c5962c-5558-405a-9cb1-5dc503d57016 None None] MFA Rules not processed for user `35cbf3fa31ef4b50a794b28752da6e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:26.916741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e8c5962c-5558-405a-9cb1-5dc503d57016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:26.998299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 829/1666] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:57:27.009477 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc918cb2-497d-4daa-b455-08d077a75634 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:27.016604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bc918cb2-497d-4daa-b455-08d077a75634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:27.134436 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:27.147100 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 838/1667] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:57:27.159244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5962c-5558-405a-9cb1-5dc503d57016 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:27.161249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e8c5962c-5558-405a-9cb1-5dc503d57016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:27.187395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc918cb2-497d-4daa-b455-08d077a75634 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:27.211095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4875e3a-5b9f-48ca-9fe0-5ba4f3669753 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:27.212516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875e3a-5b9f-48ca-9fe0-5ba4f3669753 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:27.214244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875e3a-5b9f-48ca-9fe0-5ba4f3669753 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:27.214875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875e3a-5b9f-48ca-9fe0-5ba4f3669753 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:27.248626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4875e3a-5b9f-48ca-9fe0-5ba4f3669753 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:27.252463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4875e3a-5b9f-48ca-9fe0-5ba4f3669753 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:27.258974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5962c-5558-405a-9cb1-5dc503d57016 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:27.263270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69003a0c-fc66-48a7-b001-8d476ef8c7eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:27.264198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69003a0c-fc66-48a7-b001-8d476ef8c7eb service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:27.264672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69003a0c-fc66-48a7-b001-8d476ef8c7eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:27.265114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69003a0c-fc66-48a7-b001-8d476ef8c7eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:27.276640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69003a0c-fc66-48a7-b001-8d476ef8c7eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:27.278418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69003a0c-fc66-48a7-b001-8d476ef8c7eb service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:27.283211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:27.286688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 830/1668] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:27.315825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 839/1669] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:57:27.502779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4875e3a-5b9f-48ca-9fe0-5ba4f3669753 service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:27.522226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4875e3a-5b9f-48ca-9fe0-5ba4f3669753 service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:27.540667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd3fc4-3615-4427-b023-87720150b78e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:27.541377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd3fc4-3615-4427-b023-87720150b78e service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:27.541652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd3fc4-3615-4427-b023-87720150b78e service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:27.546619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd3fc4-3615-4427-b023-87720150b78e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:27.690476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd3fc4-3615-4427-b023-87720150b78e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:27.694391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd3fc4-3615-4427-b023-87720150b78e service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:27.720202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 831/1670] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:28.879495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fb93a-60e2-41b8-9bce-3ef0c0706b37 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:28.879955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fb93a-60e2-41b8-9bce-3ef0c0706b37 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:28.880050 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fb93a-60e2-41b8-9bce-3ef0c0706b37 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:28.881481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 840/1671] 10.222.0.20 () {54 vars in 912 bytes} [Mon Jul 11 20:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:57:28.887344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120d343-a9ea-47b0-8bf4-e1e4f788e9a6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:28.888357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120d343-a9ea-47b0-8bf4-e1e4f788e9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:28.888844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120d343-a9ea-47b0-8bf4-e1e4f788e9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:28.959016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4120d343-a9ea-47b0-8bf4-e1e4f788e9a6 None None] MFA Rules not processed for user `0574d5c8974f43218653208ca72c43bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:28.960379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4120d343-a9ea-47b0-8bf4-e1e4f788e9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:29.184826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 832/1672] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:29.225546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fb93a-60e2-41b8-9bce-3ef0c0706b37 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:29.228069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-860fb93a-60e2-41b8-9bce-3ef0c0706b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:29.252305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fb93a-60e2-41b8-9bce-3ef0c0706b37 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:29.260737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e676a-f86f-4a2f-9e77-11e909484e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:29.262951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e676a-f86f-4a2f-9e77-11e909484e4b service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:29.263417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e676a-f86f-4a2f-9e77-11e909484e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:29.265643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e676a-f86f-4a2f-9e77-11e909484e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:29.287954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e676a-f86f-4a2f-9e77-11e909484e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:29.289649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e676a-f86f-4a2f-9e77-11e909484e4b service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:29.311192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 841/1673] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:29.783541 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702864fd-7d64-4eca-81c6-b63e5b2862d7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:29.783877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702864fd-7d64-4eca-81c6-b63e5b2862d7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:29.784539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702864fd-7d64-4eca-81c6-b63e5b2862d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:29.843845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-702864fd-7d64-4eca-81c6-b63e5b2862d7 None None] MFA Rules not processed for user `b5871ced3beb4decacc3a20afe523349`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:29.845312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-702864fd-7d64-4eca-81c6-b63e5b2862d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:30.032350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8350e78-cde7-4416-9de5-76243bec3b70 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:30.033115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8350e78-cde7-4416-9de5-76243bec3b70 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:30.033227 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 833/1674] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:30.033615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8350e78-cde7-4416-9de5-76243bec3b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:30.064779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c8350e78-cde7-4416-9de5-76243bec3b70 None None] MFA Rules not processed for user `b5871ced3beb4decacc3a20afe523349`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:30.066097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c8350e78-cde7-4416-9de5-76243bec3b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:30.085273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 842/1675] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:30.097637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702864fd-7d64-4eca-81c6-b63e5b2862d7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:30.101383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-702864fd-7d64-4eca-81c6-b63e5b2862d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:30.135326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702864fd-7d64-4eca-81c6-b63e5b2862d7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:30.148148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a0782-7ffe-4a17-a76a-f850c65b1138 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:30.148845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a0782-7ffe-4a17-a76a-f850c65b1138 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:30.149180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a0782-7ffe-4a17-a76a-f850c65b1138 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:30.149398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a0782-7ffe-4a17-a76a-f850c65b1138 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:30.169521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a0782-7ffe-4a17-a76a-f850c65b1138 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:30.170831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a0782-7ffe-4a17-a76a-f850c65b1138 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:30.180163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:30.204501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 834/1676] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:57:31.004207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec917351-e006-48e2-8161-9098e37c2c67 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:31.004504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec917351-e006-48e2-8161-9098e37c2c67 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:31.004786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec917351-e006-48e2-8161-9098e37c2c67 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:31.006291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 843/1677] 10.222.0.20 () {54 vars in 912 bytes} [Mon Jul 11 20:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:57:31.022446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc923ab-c862-4d23-a76e-7fc315630111 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:31.023276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc923ab-c862-4d23-a76e-7fc315630111 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:31.023765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc923ab-c862-4d23-a76e-7fc315630111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:31.084025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-efc923ab-c862-4d23-a76e-7fc315630111 None None] MFA Rules not processed for user `0574d5c8974f43218653208ca72c43bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:31.088598 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-efc923ab-c862-4d23-a76e-7fc315630111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:31.120964 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 835/1678] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 20:57:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:31.124380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec917351-e006-48e2-8161-9098e37c2c67 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:31.126360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ec917351-e006-48e2-8161-9098e37c2c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:31.162095 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec917351-e006-48e2-8161-9098e37c2c67 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:31.169826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deac44-14c2-42e8-8db9-25273a1ffab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:31.172032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deac44-14c2-42e8-8db9-25273a1ffab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:31.176480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deac44-14c2-42e8-8db9-25273a1ffab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:31.177045 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deac44-14c2-42e8-8db9-25273a1ffab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:31.213453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deac44-14c2-42e8-8db9-25273a1ffab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:31.215525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deac44-14c2-42e8-8db9-25273a1ffab4 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:31.234021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 844/1679] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:31.250870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212939f-33a7-4fda-b799-56134f4853b2 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:31.251038 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212939f-33a7-4fda-b799-56134f4853b2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:31.251408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212939f-33a7-4fda-b799-56134f4853b2 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:31.252981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 836/1680] 10.222.0.20 () {54 vars in 960 bytes} [Mon Jul 11 20:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:57:31.258837 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370a301-4034-49a9-b606-52650e83abb0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:31.259456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370a301-4034-49a9-b606-52650e83abb0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:31.259948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370a301-4034-49a9-b606-52650e83abb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:31.285408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9370a301-4034-49a9-b606-52650e83abb0 None None] MFA Rules not processed for user `946d11b552754364b55eb495f68ab852`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:31.287046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9370a301-4034-49a9-b606-52650e83abb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:31.455190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 845/1681] 10.222.0.20 () {58 vars in 1057 bytes} [Mon Jul 11 20:57:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:31.464680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212939f-33a7-4fda-b799-56134f4853b2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:31.497284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212939f-33a7-4fda-b799-56134f4853b2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:31.512543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a22bf8-7371-4835-8339-c5d2dfae8ab9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:31.513657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a22bf8-7371-4835-8339-c5d2dfae8ab9 service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:31.515852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a22bf8-7371-4835-8339-c5d2dfae8ab9 service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:31.516482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a22bf8-7371-4835-8339-c5d2dfae8ab9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:31.520420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-46a22bf8-7371-4835-8339-c5d2dfae8ab9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:31.561729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a22bf8-7371-4835-8339-c5d2dfae8ab9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:31.564952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a22bf8-7371-4835-8339-c5d2dfae8ab9 service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:31.599184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dc11a-5a58-4a9f-80e4-00c808e33fb0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:31.599793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dc11a-5a58-4a9f-80e4-00c808e33fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:31.600471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dc11a-5a58-4a9f-80e4-00c808e33fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:31.613090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 837/1682] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:31.640429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-939dc11a-5a58-4a9f-80e4-00c808e33fb0 None None] MFA Rules not processed for user `aa7e6c5842a84e91ac94c1107ad29158`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:31.642978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-939dc11a-5a58-4a9f-80e4-00c808e33fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:31.656949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 846/1683] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:32.598602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a22bf8-7371-4835-8339-c5d2dfae8ab9 service placement] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:32.602254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-46a22bf8-7371-4835-8339-c5d2dfae8ab9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:32.841754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a22bf8-7371-4835-8339-c5d2dfae8ab9 service placement] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:32.860215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:32.861635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:32.865354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:32.865705 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] PATH_INFO: `/v3/users/0b03a4195935439e923932201f357623` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:32.870398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b03a4195935439e923932201f357623)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:32.883282 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:33.037135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b03a4195935439e923932201f357623'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:33.038115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b03a4195935439e923932201f357623'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:33.072543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b03a4195935439e923932201f357623'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:33.075606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b03a4195935439e923932201f357623'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:33.128674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b03a4195935439e923932201f357623'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:33.179876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 838/1684] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:57:32 2022] DELETE /identity/v3/users/0b03a4195935439e923932201f357623 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:33.184042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc3d80-b5ea-4a8e-920d-273a3bda9d6b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:33.189928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc3d80-b5ea-4a8e-920d-273a3bda9d6b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:33.190927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc3d80-b5ea-4a8e-920d-273a3bda9d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:33.258272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d4bc3d80-b5ea-4a8e-920d-273a3bda9d6b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:33.259474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d4bc3d80-b5ea-4a8e-920d-273a3bda9d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:33.560466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 847/1685] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:33.574687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:33.577462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:33.804036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09521f18-c4e9-4d58-8995-ea6330be7369 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:33.813533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd83b1-6709-4570-babe-43afa63c46d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:33.822870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd83b1-6709-4570-babe-43afa63c46d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:33.824694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd83b1-6709-4570-babe-43afa63c46d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:33.825126 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd83b1-6709-4570-babe-43afa63c46d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:33.861075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd83b1-6709-4570-babe-43afa63c46d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:33.862558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd83b1-6709-4570-babe-43afa63c46d8 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:33.877448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:33.902397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 839/1686] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:34.040734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c761a-ccfa-447d-8cf7-6aaaf1d97451 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:34.041339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c761a-ccfa-447d-8cf7-6aaaf1d97451 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:34.041678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c761a-ccfa-447d-8cf7-6aaaf1d97451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:34.139463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fa6c761a-ccfa-447d-8cf7-6aaaf1d97451 None None] MFA Rules not processed for user `0965444614ae486e80e111f4126b52f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:34.153405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fa6c761a-ccfa-447d-8cf7-6aaaf1d97451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:34.348447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:34.409746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 848/1687] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:34.427143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd83b1-6709-4570-babe-43afa63c46d8 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:34.428576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-12dd83b1-6709-4570-babe-43afa63c46d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:34.698174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd83b1-6709-4570-babe-43afa63c46d8 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:34.737786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a91135-445c-4f70-a1e9-119fee02577d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:34.738766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a91135-445c-4f70-a1e9-119fee02577d service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:34.739200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a91135-445c-4f70-a1e9-119fee02577d service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:34.739624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a91135-445c-4f70-a1e9-119fee02577d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:34.769881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a91135-445c-4f70-a1e9-119fee02577d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:34.775168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a91135-445c-4f70-a1e9-119fee02577d service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:34.836455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 840/1688] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:35.177295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c761a-ccfa-447d-8cf7-6aaaf1d97451 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:35.178762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fa6c761a-ccfa-447d-8cf7-6aaaf1d97451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:35.202380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c761a-ccfa-447d-8cf7-6aaaf1d97451 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:35.221718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:35.223759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:35.224414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:35.227162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] PATH_INFO: `/v3/projects/6001641b8786432fbe8c94c64855139a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:35.230880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6001641b8786432fbe8c94c64855139a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:35.233684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:35.472631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6001641b8786432fbe8c94c64855139a', 'request_id': 'req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:35.486139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6001641b8786432fbe8c94c64855139a', 'request_id': 'req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:35.509660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6001641b8786432fbe8c94c64855139a', 'request_id': 'req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:35.523960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] The token cache is being invalidate because project 6001641b8786432fbe8c94c64855139a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:57:35.524555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:35.525387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6381e50e-1476-4fb9-b6c3-6c83e1badf3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:35.527522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 849/1689] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:57:35 2022] DELETE /identity/v3/projects/6001641b8786432fbe8c94c64855139a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:57:35.532677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a745-f438-475e-8090-1660bd32f5ed None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:35.533904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a745-f438-475e-8090-1660bd32f5ed None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:35.533904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a745-f438-475e-8090-1660bd32f5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:35.583062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d347a745-f438-475e-8090-1660bd32f5ed None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:35.587423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d347a745-f438-475e-8090-1660bd32f5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:35.797412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 841/1690] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:35.806255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e79d7-9b6d-4f92-9146-2f4adef72bb0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:35.806877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e79d7-9b6d-4f92-9146-2f4adef72bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:35.807690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e79d7-9b6d-4f92-9146-2f4adef72bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:35.858937 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e64e79d7-9b6d-4f92-9146-2f4adef72bb0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:35.860570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e64e79d7-9b6d-4f92-9146-2f4adef72bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:35.883199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 850/1691] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:35.887345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347a745-f438-475e-8090-1660bd32f5ed None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:35.903514 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347a745-f438-475e-8090-1660bd32f5ed None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:35.920917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5024e4-7d8f-4d5f-9a7b-41fe7344a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:35.921663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5024e4-7d8f-4d5f-9a7b-41fe7344a553 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:35.922026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5024e4-7d8f-4d5f-9a7b-41fe7344a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:35.922299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5024e4-7d8f-4d5f-9a7b-41fe7344a553 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:35.923092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5024e4-7d8f-4d5f-9a7b-41fe7344a553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:35.923760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5024e4-7d8f-4d5f-9a7b-41fe7344a553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:35.925672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5024e4-7d8f-4d5f-9a7b-41fe7344a553 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:35.958254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 842/1692] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:35.966851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e79d7-9b6d-4f92-9146-2f4adef72bb0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:35.981092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e79d7-9b6d-4f92-9146-2f4adef72bb0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:36.001105 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6169e-5af7-4a1f-90e8-0096fbe3192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:36.001811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6169e-5af7-4a1f-90e8-0096fbe3192b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:36.002089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6169e-5af7-4a1f-90e8-0096fbe3192b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:36.002360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6169e-5af7-4a1f-90e8-0096fbe3192b admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:36.003071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6169e-5af7-4a1f-90e8-0096fbe3192b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:36.005163 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6169e-5af7-4a1f-90e8-0096fbe3192b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:36.047394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 851/1693] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:35 2022] POST /identity/v3/projects => generated 365 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:36.055592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e916b-2419-4c02-98e7-70c2741e5a22 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:36.055883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e916b-2419-4c02-98e7-70c2741e5a22 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:36.056173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e916b-2419-4c02-98e7-70c2741e5a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:36.057973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6169e-5af7-4a1f-90e8-0096fbe3192b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:36.103709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-fd3e916b-2419-4c02-98e7-70c2741e5a22 None None] MFA Rules not processed for user `77cf43c378a9413dbd799a00bfa21eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:36.105567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fd3e916b-2419-4c02-98e7-70c2741e5a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:36.202147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6169e-5af7-4a1f-90e8-0096fbe3192b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:36.210261 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cee52f-fbf0-4397-b19f-b1a4063a38b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:36.212594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cee52f-fbf0-4397-b19f-b1a4063a38b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:36.212921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cee52f-fbf0-4397-b19f-b1a4063a38b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:36.213184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cee52f-fbf0-4397-b19f-b1a4063a38b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:36.213924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cee52f-fbf0-4397-b19f-b1a4063a38b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:36.217245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cee52f-fbf0-4397-b19f-b1a4063a38b3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:36.262251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 852/1694] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:36 2022] POST /identity/v3/users => generated 283 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:36.273812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cee52f-fbf0-4397-b19f-b1a4063a38b3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:36.295008 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 843/1695] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:57:36.295319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cee52f-fbf0-4397-b19f-b1a4063a38b3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:36.306829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e916b-2419-4c02-98e7-70c2741e5a22 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:36.309536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fd3e916b-2419-4c02-98e7-70c2741e5a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:36.315285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:36.318006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e8404-26d2-4fbc-b8b9-a6da8bf1473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:36.320503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e8404-26d2-4fbc-b8b9-a6da8bf1473c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:36.322961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e8404-26d2-4fbc-b8b9-a6da8bf1473c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:36.323247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e8404-26d2-4fbc-b8b9-a6da8bf1473c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:36.324022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e8404-26d2-4fbc-b8b9-a6da8bf1473c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:36.327591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e8404-26d2-4fbc-b8b9-a6da8bf1473c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:36.363649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 853/1696] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:36.365377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e8404-26d2-4fbc-b8b9-a6da8bf1473c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:36.386859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e8404-26d2-4fbc-b8b9-a6da8bf1473c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:36.394516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1849d7-cf5b-4867-9da0-98208215c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:36.396988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1849d7-cf5b-4867-9da0-98208215c4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:36.397480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1849d7-cf5b-4867-9da0-98208215c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:36.397965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1849d7-cf5b-4867-9da0-98208215c4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:36.398878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1849d7-cf5b-4867-9da0-98208215c4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:36.400897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1849d7-cf5b-4867-9da0-98208215c4e7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:36.413714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 854/1697] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:36.417851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1849d7-cf5b-4867-9da0-98208215c4e7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:36.430102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1849d7-cf5b-4867-9da0-98208215c4e7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:36.436994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c0dea-6d79-40a4-8bd8-15e9617f7ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:36.438183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0dea-6d79-40a4-8bd8-15e9617f7ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:36.438689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0dea-6d79-40a4-8bd8-15e9617f7ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:36.439179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0dea-6d79-40a4-8bd8-15e9617f7ca9 admin admin] PATH_INFO: `/v3/projects/3b0d3472514e490184dfa15f661773fc/users/5e0dc3493bcf47be966e77d6c43b7fff/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:36.442515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:36.463650 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e916b-2419-4c02-98e7-70c2741e5a22 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:36.465995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0dea-6d79-40a4-8bd8-15e9617f7ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0d3472514e490184dfa15f661773fc, user_id=5e0dc3493bcf47be966e77d6c43b7fff, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:36.468891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0dea-6d79-40a4-8bd8-15e9617f7ca9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:36.476528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc3fad-ba79-41f6-8221-ffcba4ece695 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:36.477530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc3fad-ba79-41f6-8221-ffcba4ece695 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:36.478042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc3fad-ba79-41f6-8221-ffcba4ece695 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:36.478498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc3fad-ba79-41f6-8221-ffcba4ece695 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:36.488283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 855/1698] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:36 2022] PUT /identity/v3/projects/3b0d3472514e490184dfa15f661773fc/users/5e0dc3493bcf47be966e77d6c43b7fff/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:36.489935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc3fad-ba79-41f6-8221-ffcba4ece695 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:36.491974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc3fad-ba79-41f6-8221-ffcba4ece695 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:36.498989 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa75bb-c450-4e80-ab4e-852ea5e3fa16 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:36.499386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa75bb-c450-4e80-ab4e-852ea5e3fa16 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:36.499662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa75bb-c450-4e80-ab4e-852ea5e3fa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:36.505336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 844/1699] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:36.532275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-37aa75bb-c450-4e80-ab4e-852ea5e3fa16 None None] MFA Rules not processed for user `5e0dc3493bcf47be966e77d6c43b7fff`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:36.533359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-37aa75bb-c450-4e80-ab4e-852ea5e3fa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:36.619649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 856/1700] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:36.632723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc3fad-ba79-41f6-8221-ffcba4ece695 service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:36.634182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-45fc3fad-ba79-41f6-8221-ffcba4ece695 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:36.712985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc3fad-ba79-41f6-8221-ffcba4ece695 service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:36.789647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b546ed-6f9a-45a8-b762-91a5d976db81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:36.790321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b546ed-6f9a-45a8-b762-91a5d976db81 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:36.790748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b546ed-6f9a-45a8-b762-91a5d976db81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:36.790902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b546ed-6f9a-45a8-b762-91a5d976db81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:36.800881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b546ed-6f9a-45a8-b762-91a5d976db81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:36.802516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b546ed-6f9a-45a8-b762-91a5d976db81 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:36.825297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 845/1701] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:37.919689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383663a-46c5-4194-a6b8-701759ba60f2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:37.920233 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383663a-46c5-4194-a6b8-701759ba60f2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:37.921353 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383663a-46c5-4194-a6b8-701759ba60f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:37.963229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1383663a-46c5-4194-a6b8-701759ba60f2 None None] MFA Rules not processed for user `0965444614ae486e80e111f4126b52f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:37.964646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1383663a-46c5-4194-a6b8-701759ba60f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:38.061833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 857/1702] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:39.311201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcccc22-88d2-484a-8e3a-1627712ad8e0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:39.312068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcccc22-88d2-484a-8e3a-1627712ad8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:39.312513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcccc22-88d2-484a-8e3a-1627712ad8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:39.373624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0bcccc22-88d2-484a-8e3a-1627712ad8e0 None None] MFA Rules not processed for user `77cf43c378a9413dbd799a00bfa21eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:39.376651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0bcccc22-88d2-484a-8e3a-1627712ad8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:39.449302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:39.485258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 846/1703] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:41.184948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba716e-f5e2-4f11-9a9e-9f0d22b0c07d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:41.185502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba716e-f5e2-4f11-9a9e-9f0d22b0c07d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:41.185502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba716e-f5e2-4f11-9a9e-9f0d22b0c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:41.380630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-15ba716e-f5e2-4f11-9a9e-9f0d22b0c07d None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:41.400057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-15ba716e-f5e2-4f11-9a9e-9f0d22b0c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:41.577243 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:41.636598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 858/1704] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 20:57:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:41.940556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcccc22-88d2-484a-8e3a-1627712ad8e0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:41.942512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0bcccc22-88d2-484a-8e3a-1627712ad8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:42.273679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcccc22-88d2-484a-8e3a-1627712ad8e0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:42.288949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422a961-f00c-462b-9342-af2f66356063 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:42.289657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422a961-f00c-462b-9342-af2f66356063 service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:42.290017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422a961-f00c-462b-9342-af2f66356063 service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:42.290293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422a961-f00c-462b-9342-af2f66356063 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:42.341574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422a961-f00c-462b-9342-af2f66356063 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:42.344148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422a961-f00c-462b-9342-af2f66356063 service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:42.396645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 847/1705] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 20:57:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 20:57:44.101601 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75c68d-7c76-4ed2-b2e9-d010da34a166 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:44.102844 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75c68d-7c76-4ed2-b2e9-d010da34a166 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:44.103297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75c68d-7c76-4ed2-b2e9-d010da34a166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:44.168825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3b75c68d-7c76-4ed2-b2e9-d010da34a166 None None] MFA Rules not processed for user `0574d5c8974f43218653208ca72c43bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:44.171309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3b75c68d-7c76-4ed2-b2e9-d010da34a166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:44.419697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 859/1706] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jul 11 20:57:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:44.584618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422a961-f00c-462b-9342-af2f66356063 service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:44.586466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e422a961-f00c-462b-9342-af2f66356063 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:44.610950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422a961-f00c-462b-9342-af2f66356063 service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:44.627002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cafa43-9519-4cad-adba-5ba249220f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:44.627743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cafa43-9519-4cad-adba-5ba249220f03 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:44.628016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cafa43-9519-4cad-adba-5ba249220f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:44.629064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cafa43-9519-4cad-adba-5ba249220f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:44.662086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cafa43-9519-4cad-adba-5ba249220f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:44.663614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cafa43-9519-4cad-adba-5ba249220f03 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:44.691151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 848/1707] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:44.872696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75c68d-7c76-4ed2-b2e9-d010da34a166 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:44.874309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3b75c68d-7c76-4ed2-b2e9-d010da34a166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:45.106897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75c68d-7c76-4ed2-b2e9-d010da34a166 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:45.130357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:45.131451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:45.131738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:45.132068 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] PATH_INFO: `/v3/users/cca4ef98735c4c84bbf61b9371ee33a4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:45.136298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cca4ef98735c4c84bbf61b9371ee33a4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:45.136542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:45.264466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cca4ef98735c4c84bbf61b9371ee33a4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:45.286237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cca4ef98735c4c84bbf61b9371ee33a4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:45.288824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cca4ef98735c4c84bbf61b9371ee33a4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:45.322068 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cca4ef98735c4c84bbf61b9371ee33a4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:45.370932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cca4ef98735c4c84bbf61b9371ee33a4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:45.390481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 860/1708] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:57:44 2022] DELETE /identity/v3/users/cca4ef98735c4c84bbf61b9371ee33a4 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:45.400496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa44fd-2d2b-42f5-8b34-2312075881f6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:45.402029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa44fd-2d2b-42f5-8b34-2312075881f6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:45.402524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa44fd-2d2b-42f5-8b34-2312075881f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:45.465386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1dfa44fd-2d2b-42f5-8b34-2312075881f6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:45.470747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1dfa44fd-2d2b-42f5-8b34-2312075881f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:45.798587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 849/1709] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:45.875704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:45.877271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:45.935019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:46.074066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d96ace-f564-4202-8486-5200fbf1345e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:46.089038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5513b2-ef9a-4151-9c22-ba8534355089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:46.090726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5513b2-ef9a-4151-9c22-ba8534355089 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:46.091214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5513b2-ef9a-4151-9c22-ba8534355089 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:46.091699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5513b2-ef9a-4151-9c22-ba8534355089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:46.135436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5513b2-ef9a-4151-9c22-ba8534355089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:46.141862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5513b2-ef9a-4151-9c22-ba8534355089 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:46.174665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 861/1710] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:49.731827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa44fd-2d2b-42f5-8b34-2312075881f6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:49.733339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1dfa44fd-2d2b-42f5-8b34-2312075881f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:49.770199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa44fd-2d2b-42f5-8b34-2312075881f6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:49.784180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:49.787666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:49.789659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:49.791431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] PATH_INFO: `/v3/projects/f93db91e0926470da7fc7446540e98e6` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:49.798699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93db91e0926470da7fc7446540e98e6)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:49.805283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 20:57:49.805283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Jul 11 20:57:49.805283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: Jul 11 20:57:49.809823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:50.012377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:50.030049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93db91e0926470da7fc7446540e98e6', 'request_id': 'req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:50.060617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93db91e0926470da7fc7446540e98e6', 'request_id': 'req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:50.094456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93db91e0926470da7fc7446540e98e6', 'request_id': 'req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:50.130783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] The token cache is being invalidate because project f93db91e0926470da7fc7446540e98e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:57:50.131612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:50.135160 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:50.140327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 850/1711] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:57:49 2022] DELETE /identity/v3/projects/f93db91e0926470da7fc7446540e98e6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:50.144716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5513b2-ef9a-4151-9c22-ba8534355089 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:50.172785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5513b2-ef9a-4151-9c22-ba8534355089 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:50.460421 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:50.461514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:50.462549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:50.466167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] PATH_INFO: `/v3/users/35cbf3fa31ef4b50a794b28752da6e56` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:50.468020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=35cbf3fa31ef4b50a794b28752da6e56)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:50.470380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:50.551882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35cbf3fa31ef4b50a794b28752da6e56'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:50.561062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35cbf3fa31ef4b50a794b28752da6e56'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:50.562135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35cbf3fa31ef4b50a794b28752da6e56'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:50.576227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35cbf3fa31ef4b50a794b28752da6e56'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:50.603028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8f9d549a-f3d0-48f2-9867-cb03b03d3a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35cbf3fa31ef4b50a794b28752da6e56'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:50.622325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 862/1712] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:57:50 2022] DELETE /identity/v3/users/35cbf3fa31ef4b50a794b28752da6e56 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:50.930498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:50.932534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:50.993980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed1363-065c-48cd-a82c-63bd2c6eb21e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:51.257353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:51.260187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:51.261134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:51.263242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] PATH_INFO: `/v3/users/907c2ffd4b5e4292a4fae1bf11e53475` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:51.265700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=907c2ffd4b5e4292a4fae1bf11e53475)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:51.274187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:51.408402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '907c2ffd4b5e4292a4fae1bf11e53475'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:51.410460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '907c2ffd4b5e4292a4fae1bf11e53475'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:51.435673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '907c2ffd4b5e4292a4fae1bf11e53475'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:51.437625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '907c2ffd4b5e4292a4fae1bf11e53475'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:51.474011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1c4546fb-3605-4a36-9777-ff43883173e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '907c2ffd4b5e4292a4fae1bf11e53475'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:51.506464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 851/1713] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:57:50 2022] DELETE /identity/v3/users/907c2ffd4b5e4292a4fae1bf11e53475 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:51.509870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b2552-48b6-403f-a669-dd8702bb3a78 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:51.510325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b2552-48b6-403f-a669-dd8702bb3a78 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:51.510900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b2552-48b6-403f-a669-dd8702bb3a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:51.579650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-073b2552-48b6-403f-a669-dd8702bb3a78 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:51.580916 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-073b2552-48b6-403f-a669-dd8702bb3a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:51.716642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff3b4e-daf0-44eb-b8b2-aaf9f8d61f39 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:51.717017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff3b4e-daf0-44eb-b8b2-aaf9f8d61f39 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:51.717211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff3b4e-daf0-44eb-b8b2-aaf9f8d61f39 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:51.719308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 852/1714] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 20:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:57:51.793114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:51.904084 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 863/1715] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:52.031279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af10533-63f1-4ef3-afff-f326b83b1e78 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:52.031731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af10533-63f1-4ef3-afff-f326b83b1e78 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:52.031731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af10533-63f1-4ef3-afff-f326b83b1e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:52.075001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7af10533-63f1-4ef3-afff-f326b83b1e78 None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:52.076307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7af10533-63f1-4ef3-afff-f326b83b1e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:52.124495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b2552-48b6-403f-a669-dd8702bb3a78 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:52.126053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-073b2552-48b6-403f-a669-dd8702bb3a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:52.296220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:52.348967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b2552-48b6-403f-a669-dd8702bb3a78 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:52.357491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0de00-63f2-454b-afaa-d86ddddcddbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:52.358361 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0de00-63f2-454b-afaa-d86ddddcddbd service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:52.358491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0de00-63f2-454b-afaa-d86ddddcddbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:52.358734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0de00-63f2-454b-afaa-d86ddddcddbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:52.366328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 853/1716] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 20:57:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:52.374025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0de00-63f2-454b-afaa-d86ddddcddbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:52.375723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0de00-63f2-454b-afaa-d86ddddcddbd service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:52.391089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 864/1717] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:52.726189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf38f26-b22d-47a8-bcbb-a05730dfffae None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:52.726852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf38f26-b22d-47a8-bcbb-a05730dfffae None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:52.728490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf38f26-b22d-47a8-bcbb-a05730dfffae None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:52.730784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 854/1718] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 20:57:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:57:52.965984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a607-d2e6-4b7a-96cd-37bfaa942f6f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:52.966531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a607-d2e6-4b7a-96cd-37bfaa942f6f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:52.967677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a607-d2e6-4b7a-96cd-37bfaa942f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:53.005736 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4b15a607-d2e6-4b7a-96cd-37bfaa942f6f None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:53.012005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4b15a607-d2e6-4b7a-96cd-37bfaa942f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:53.037397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 865/1719] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 20:57:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:53.962539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf38f26-b22d-47a8-bcbb-a05730dfffae None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:53.965504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6bf38f26-b22d-47a8-bcbb-a05730dfffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:54.000853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf38f26-b22d-47a8-bcbb-a05730dfffae None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:54.017684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d53c19-0b91-4282-80bb-221f7579a23f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:54.019183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d53c19-0b91-4282-80bb-221f7579a23f service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:54.019689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d53c19-0b91-4282-80bb-221f7579a23f service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:54.020189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d53c19-0b91-4282-80bb-221f7579a23f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:54.035101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d53c19-0b91-4282-80bb-221f7579a23f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:54.037117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d53c19-0b91-4282-80bb-221f7579a23f service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:54.056941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 855/1720] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:54.230857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a607-d2e6-4b7a-96cd-37bfaa942f6f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:54.271450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a607-d2e6-4b7a-96cd-37bfaa942f6f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:54.280642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae81a94-abc9-4d90-b3e8-5f219d1f373d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:54.281331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae81a94-abc9-4d90-b3e8-5f219d1f373d service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:54.281995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae81a94-abc9-4d90-b3e8-5f219d1f373d service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:54.281995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae81a94-abc9-4d90-b3e8-5f219d1f373d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:54.310529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae81a94-abc9-4d90-b3e8-5f219d1f373d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:54.312246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae81a94-abc9-4d90-b3e8-5f219d1f373d service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:54.373702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 866/1721] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:54.385200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d53c19-0b91-4282-80bb-221f7579a23f service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:54.386815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-67d53c19-0b91-4282-80bb-221f7579a23f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:54.435449 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d53c19-0b91-4282-80bb-221f7579a23f service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:54.450051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:54.451131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:54.451526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:54.451900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] PATH_INFO: `/v3/projects/24987b91a2824f08ab36c532c3add1d1` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:54.455214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24987b91a2824f08ab36c532c3add1d1)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:54.464549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:54.625571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae81a94-abc9-4d90-b3e8-5f219d1f373d service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:54.627312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2ae81a94-abc9-4d90-b3e8-5f219d1f373d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:54.685436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae81a94-abc9-4d90-b3e8-5f219d1f373d service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:54.724827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24987b91a2824f08ab36c532c3add1d1', 'request_id': 'req-6d57e22c-3cdb-483a-bdb5-083b8db37652'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:54.741508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24987b91a2824f08ab36c532c3add1d1', 'request_id': 'req-6d57e22c-3cdb-483a-bdb5-083b8db37652'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:54.762797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24987b91a2824f08ab36c532c3add1d1', 'request_id': 'req-6d57e22c-3cdb-483a-bdb5-083b8db37652'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:54.785460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] The token cache is being invalidate because project 24987b91a2824f08ab36c532c3add1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:57:54.785989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:54.787049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d57e22c-3cdb-483a-bdb5-083b8db37652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:54.789353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 856/1722] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:57:54 2022] DELETE /identity/v3/projects/24987b91a2824f08ab36c532c3add1d1 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:54.794615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559cd4b-c486-4ec8-8728-3b8dd63570b2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:54.795106 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559cd4b-c486-4ec8-8728-3b8dd63570b2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:54.795510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559cd4b-c486-4ec8-8728-3b8dd63570b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:54.837561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f559cd4b-c486-4ec8-8728-3b8dd63570b2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:54.841677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f559cd4b-c486-4ec8-8728-3b8dd63570b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:54.900289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:54.902039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:54.904055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:54.904379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] PATH_INFO: `/v3/projects/625074600ce14680b0f41382f409a450` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:54.905949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=625074600ce14680b0f41382f409a450)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:54.912010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:54.964065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 857/1723] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:54.970982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dac681-f70f-45e8-9556-f53f0e685ac5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:54.971928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dac681-f70f-45e8-9556-f53f0e685ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:54.972511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dac681-f70f-45e8-9556-f53f0e685ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:55.014631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e7dac681-f70f-45e8-9556-f53f0e685ac5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:55.015947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e7dac681-f70f-45e8-9556-f53f0e685ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:55.044785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 858/1724] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:55.051818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dac681-f70f-45e8-9556-f53f0e685ac5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:55.104433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dac681-f70f-45e8-9556-f53f0e685ac5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:55.136605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625074600ce14680b0f41382f409a450', 'request_id': 'req-48402ac9-21ba-4572-b5b2-721c252248e9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:55.150844 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625074600ce14680b0f41382f409a450', 'request_id': 'req-48402ac9-21ba-4572-b5b2-721c252248e9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:55.179586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625074600ce14680b0f41382f409a450', 'request_id': 'req-48402ac9-21ba-4572-b5b2-721c252248e9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:55.203231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:55.203568 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:55.230659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] The token cache is being invalidate because project 625074600ce14680b0f41382f409a450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:57:55.230962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:55.235430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:55.235776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3d7e8-3dc8-40c7-88ed-e5ad481df6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:55.236719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d7e8-3dc8-40c7-88ed-e5ad481df6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:55.240036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 867/1725] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:57:54 2022] DELETE /identity/v3/projects/625074600ce14680b0f41382f409a450 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:57:55.240901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d7e8-3dc8-40c7-88ed-e5ad481df6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:55.240901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d7e8-3dc8-40c7-88ed-e5ad481df6ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:55.241087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:55.241404 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3d7e8-3dc8-40c7-88ed-e5ad481df6ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:55.241735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3d7e8-3dc8-40c7-88ed-e5ad481df6ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:55.242621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:55.243414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3d7e8-3dc8-40c7-88ed-e5ad481df6ec admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:55.271827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 859/1726] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:55.272844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3d7e8-3dc8-40c7-88ed-e5ad481df6ec admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:55.281789 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-89c3d7e8-3dc8-40c7-88ed-e5ad481df6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:55.294036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48402ac9-21ba-4572-b5b2-721c252248e9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:55.307748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3d7e8-3dc8-40c7-88ed-e5ad481df6ec admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:55.319006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69163449-8c8e-4eae-ae4d-7f58d13a7d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:55.319376 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:55.319694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69163449-8c8e-4eae-ae4d-7f58d13a7d63 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:55.319974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69163449-8c8e-4eae-ae4d-7f58d13a7d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:55.320263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:55.320373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69163449-8c8e-4eae-ae4d-7f58d13a7d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:55.320564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:55.320815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] PATH_INFO: `/v3/users/e5f41154e61f47e8ba7a9830b18f827f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:55.320998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69163449-8c8e-4eae-ae4d-7f58d13a7d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:55.322939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69163449-8c8e-4eae-ae4d-7f58d13a7d63 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:55.328451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f41154e61f47e8ba7a9830b18f827f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:55.330463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:55.383183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 860/1727] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:55 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:57:55.384367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69163449-8c8e-4eae-ae4d-7f58d13a7d63 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:55.490130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f41154e61f47e8ba7a9830b18f827f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:55.511156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f41154e61f47e8ba7a9830b18f827f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:55.512643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f41154e61f47e8ba7a9830b18f827f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:55.533325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f41154e61f47e8ba7a9830b18f827f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:55.556592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f41154e61f47e8ba7a9830b18f827f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:55.572980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69163449-8c8e-4eae-ae4d-7f58d13a7d63 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:55.586196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 868/1728] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:57:55 2022] DELETE /identity/v3/users/e5f41154e61f47e8ba7a9830b18f827f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:55.593043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188c1a6-b02e-4470-9016-c918fcfa9825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:55.593758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188c1a6-b02e-4470-9016-c918fcfa9825 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:55.594042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188c1a6-b02e-4470-9016-c918fcfa9825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:55.594318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188c1a6-b02e-4470-9016-c918fcfa9825 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:55.595038 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188c1a6-b02e-4470-9016-c918fcfa9825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:55.605507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188c1a6-b02e-4470-9016-c918fcfa9825 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:55.660161 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 861/1729] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:55 2022] POST /identity/v3/users => generated 294 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:57:55.661609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:55.664625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:55.866487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658c107-b63d-4ea9-a518-6edc978ef8b0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:55.876794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ad464-5fb7-45c9-8587-12447916ddb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:55.877498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad464-5fb7-45c9-8587-12447916ddb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:55.878006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad464-5fb7-45c9-8587-12447916ddb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:55.878290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad464-5fb7-45c9-8587-12447916ddb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:55.879038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ad464-5fb7-45c9-8587-12447916ddb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:55.881092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ad464-5fb7-45c9-8587-12447916ddb7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:55.907147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 869/1730] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:55.909214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188c1a6-b02e-4470-9016-c918fcfa9825 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:55.938849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188c1a6-b02e-4470-9016-c918fcfa9825 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:55.955760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c936e-8e29-4bd8-95cc-5cca6ea86746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:55.958782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c936e-8e29-4bd8-95cc-5cca6ea86746 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:55.959427 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c936e-8e29-4bd8-95cc-5cca6ea86746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:55.961862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c936e-8e29-4bd8-95cc-5cca6ea86746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:55.965139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c936e-8e29-4bd8-95cc-5cca6ea86746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:55.975908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c936e-8e29-4bd8-95cc-5cca6ea86746 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:56.001998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 862/1731] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:56.007018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ad464-5fb7-45c9-8587-12447916ddb7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:56.025110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ad464-5fb7-45c9-8587-12447916ddb7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:56.048959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b219a63-706a-4a05-aae9-a2057e9d1419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:56.050541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b219a63-706a-4a05-aae9-a2057e9d1419 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:56.050844 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b219a63-706a-4a05-aae9-a2057e9d1419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:56.051121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b219a63-706a-4a05-aae9-a2057e9d1419 admin admin] PATH_INFO: `/v3/projects/842dfb5005414bb8a36a395189a80d26/users/a47d0a4c80564e1c848c4b87fd661f36/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:56.094607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b219a63-706a-4a05-aae9-a2057e9d1419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=842dfb5005414bb8a36a395189a80d26, user_id=a47d0a4c80564e1c848c4b87fd661f36, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:56.097739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b219a63-706a-4a05-aae9-a2057e9d1419 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:56.133007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 870/1732] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:56 2022] PUT /identity/v3/projects/842dfb5005414bb8a36a395189a80d26/users/a47d0a4c80564e1c848c4b87fd661f36/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:56.134976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f233b-476e-45f3-8ab9-3f4e515c4d4b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:56.135876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f233b-476e-45f3-8ab9-3f4e515c4d4b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:56.136357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f233b-476e-45f3-8ab9-3f4e515c4d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:56.222195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-849f233b-476e-45f3-8ab9-3f4e515c4d4b None None] MFA Rules not processed for user `a47d0a4c80564e1c848c4b87fd661f36`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:56.223447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-849f233b-476e-45f3-8ab9-3f4e515c4d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:56.408653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 863/1733] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:56.414485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b219a63-706a-4a05-aae9-a2057e9d1419 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:56.477857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:56.513230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b219a63-706a-4a05-aae9-a2057e9d1419 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:56.518239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9def7-1ebe-4c58-9d6c-449741423f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:56.519496 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9def7-1ebe-4c58-9d6c-449741423f8e admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:56.519834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9def7-1ebe-4c58-9d6c-449741423f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:56.520206 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9def7-1ebe-4c58-9d6c-449741423f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:56.520978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9def7-1ebe-4c58-9d6c-449741423f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:56.523614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9def7-1ebe-4c58-9d6c-449741423f8e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:56.555588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 871/1734] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:56 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:56.558273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f233b-476e-45f3-8ab9-3f4e515c4d4b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:56.574309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f233b-476e-45f3-8ab9-3f4e515c4d4b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:56.615205 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:56.658795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d43560-6e3a-4cfc-a514-778fcd5e4f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:56.659611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d43560-6e3a-4cfc-a514-778fcd5e4f45 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:56.659962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d43560-6e3a-4cfc-a514-778fcd5e4f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:56.660292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d43560-6e3a-4cfc-a514-778fcd5e4f45 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:56.660986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d43560-6e3a-4cfc-a514-778fcd5e4f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:56.662942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d43560-6e3a-4cfc-a514-778fcd5e4f45 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:56.688262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 864/1735] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:56 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:57:56.689898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9def7-1ebe-4c58-9d6c-449741423f8e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:56.701155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9def7-1ebe-4c58-9d6c-449741423f8e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:56.706057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe13db3-2bd4-4ea9-a37c-1704105679de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:56.706820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe13db3-2bd4-4ea9-a37c-1704105679de admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:56.707099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe13db3-2bd4-4ea9-a37c-1704105679de admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:56.707372 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe13db3-2bd4-4ea9-a37c-1704105679de admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:56.708135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe13db3-2bd4-4ea9-a37c-1704105679de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:56.709778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe13db3-2bd4-4ea9-a37c-1704105679de admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:56.722098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 872/1736] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:57:56.726684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d43560-6e3a-4cfc-a514-778fcd5e4f45 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:56.738244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d43560-6e3a-4cfc-a514-778fcd5e4f45 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:56.743275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044c0fe-be8b-4593-ad1b-eb34c9cabdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:56.744231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044c0fe-be8b-4593-ad1b-eb34c9cabdbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:56.744504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044c0fe-be8b-4593-ad1b-eb34c9cabdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:56.744774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044c0fe-be8b-4593-ad1b-eb34c9cabdbb admin admin] PATH_INFO: `/v3/projects/b523284a58444be89d5bc58cc2697a76/users/147706b62e1d4e64a9ff53cdd32a3059/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:56.762377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044c0fe-be8b-4593-ad1b-eb34c9cabdbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b523284a58444be89d5bc58cc2697a76, user_id=147706b62e1d4e64a9ff53cdd32a3059, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:56.765119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044c0fe-be8b-4593-ad1b-eb34c9cabdbb admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:56.780348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 865/1737] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:56 2022] PUT /identity/v3/projects/b523284a58444be89d5bc58cc2697a76/users/147706b62e1d4e64a9ff53cdd32a3059/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:56.784519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe13db3-2bd4-4ea9-a37c-1704105679de admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:56.795052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe13db3-2bd4-4ea9-a37c-1704105679de admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:56.857619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429a863-a43b-4506-ba86-1aad5bb8c6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:56.858310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429a863-a43b-4506-ba86-1aad5bb8c6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:56.858591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429a863-a43b-4506-ba86-1aad5bb8c6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:56.858971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429a863-a43b-4506-ba86-1aad5bb8c6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:56.859601 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429a863-a43b-4506-ba86-1aad5bb8c6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:56.861264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429a863-a43b-4506-ba86-1aad5bb8c6f2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:56.871809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 873/1738] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:56.875930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044c0fe-be8b-4593-ad1b-eb34c9cabdbb admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:56.887455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044c0fe-be8b-4593-ad1b-eb34c9cabdbb admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:56.892383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f7b54-ff71-4a53-8a5c-7c30819a43c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:56.893429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f7b54-ff71-4a53-8a5c-7c30819a43c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:56.893829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f7b54-ff71-4a53-8a5c-7c30819a43c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:56.894209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f7b54-ff71-4a53-8a5c-7c30819a43c7 admin admin] PATH_INFO: `/v3/projects/b523284a58444be89d5bc58cc2697a76/users/147706b62e1d4e64a9ff53cdd32a3059/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:56.896376 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429a863-a43b-4506-ba86-1aad5bb8c6f2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:56.897992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f7b54-ff71-4a53-8a5c-7c30819a43c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b523284a58444be89d5bc58cc2697a76, user_id=147706b62e1d4e64a9ff53cdd32a3059, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:56.898177 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9429a863-a43b-4506-ba86-1aad5bb8c6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:56.901157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f7b54-ff71-4a53-8a5c-7c30819a43c7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:56.913309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429a863-a43b-4506-ba86-1aad5bb8c6f2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:56.916863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 866/1739] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:56 2022] PUT /identity/v3/projects/b523284a58444be89d5bc58cc2697a76/users/147706b62e1d4e64a9ff53cdd32a3059/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:57:56.927380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88350f-05bc-4e97-89db-26b999be3d5f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:56.927740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88350f-05bc-4e97-89db-26b999be3d5f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:56.928095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88350f-05bc-4e97-89db-26b999be3d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:56.955003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-be88350f-05bc-4e97-89db-26b999be3d5f None None] MFA Rules not processed for user `147706b62e1d4e64a9ff53cdd32a3059`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:56.956409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-be88350f-05bc-4e97-89db-26b999be3d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:56.972530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:56.973407 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:56.973687 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:56.973964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] PATH_INFO: `/v3/projects/64b8e670b8c2436fa7e70a22be497af6` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:56.975251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64b8e670b8c2436fa7e70a22be497af6)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:56.977326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.027023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 867/1740] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:57:57.030315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64b8e670b8c2436fa7e70a22be497af6', 'request_id': 'req-65133de1-53ca-4b1e-b33e-ce0df2037033'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:57.037853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:57.039020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88350f-05bc-4e97-89db-26b999be3d5f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:57.040441 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-be88350f-05bc-4e97-89db-26b999be3d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:57.048178 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64b8e670b8c2436fa7e70a22be497af6', 'request_id': 'req-65133de1-53ca-4b1e-b33e-ce0df2037033'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:57.058440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64b8e670b8c2436fa7e70a22be497af6', 'request_id': 'req-65133de1-53ca-4b1e-b33e-ce0df2037033'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:57.071059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] The token cache is being invalidate because project 64b8e670b8c2436fa7e70a22be497af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:57:57.071701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:57.072444 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-65133de1-53ca-4b1e-b33e-ce0df2037033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:57.074583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 874/1741] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:57:56 2022] DELETE /identity/v3/projects/64b8e670b8c2436fa7e70a22be497af6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:57:57.079037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d780be-9cbb-4883-bb9d-9a56adcaafea None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.079329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d780be-9cbb-4883-bb9d-9a56adcaafea None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.079607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d780be-9cbb-4883-bb9d-9a56adcaafea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.098422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-28d780be-9cbb-4883-bb9d-9a56adcaafea None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:57.099631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-28d780be-9cbb-4883-bb9d-9a56adcaafea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:57.107811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88350f-05bc-4e97-89db-26b999be3d5f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:57.125284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:57.168804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc91b-7cd4-48af-b4e4-f2aa23be0c4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:57.169474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc91b-7cd4-48af-b4e4-f2aa23be0c4e service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.169758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc91b-7cd4-48af-b4e4-f2aa23be0c4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.170030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc91b-7cd4-48af-b4e4-f2aa23be0c4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.171905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-79ecc91b-7cd4-48af-b4e4-f2aa23be0c4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:57.173547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 875/1742] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:57.182778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa92e2-ddb6-495e-8aa9-2109516cbe9b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.183065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa92e2-ddb6-495e-8aa9-2109516cbe9b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.183509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa92e2-ddb6-495e-8aa9-2109516cbe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.203387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-32aa92e2-ddb6-495e-8aa9-2109516cbe9b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:57.204622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-32aa92e2-ddb6-495e-8aa9-2109516cbe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:57.213090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 876/1743] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:57:57.217065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa92e2-ddb6-495e-8aa9-2109516cbe9b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:57.227398 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa92e2-ddb6-495e-8aa9-2109516cbe9b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:57.233447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2d1c-ab72-458e-9bcf-bec89e272824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:57.234346 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2d1c-ab72-458e-9bcf-bec89e272824 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.234636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2d1c-ab72-458e-9bcf-bec89e272824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.234915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2d1c-ab72-458e-9bcf-bec89e272824 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.235658 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd2d1c-ab72-458e-9bcf-bec89e272824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:57.235966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd2d1c-ab72-458e-9bcf-bec89e272824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:57.238104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd2d1c-ab72-458e-9bcf-bec89e272824 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.248657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc91b-7cd4-48af-b4e4-f2aa23be0c4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:57.250117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc91b-7cd4-48af-b4e4-f2aa23be0c4e service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.257182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 877/1744] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:57:57.261936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2d1c-ab72-458e-9bcf-bec89e272824 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:57.271127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 868/1745] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:57:57.274812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2d1c-ab72-458e-9bcf-bec89e272824 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:57.280868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3bcbf-2f32-49e5-99af-12b99e9078a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:57.281707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3bcbf-2f32-49e5-99af-12b99e9078a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.281996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3bcbf-2f32-49e5-99af-12b99e9078a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.282274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3bcbf-2f32-49e5-99af-12b99e9078a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.283339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3bcbf-2f32-49e5-99af-12b99e9078a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:57.285504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3bcbf-2f32-49e5-99af-12b99e9078a8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.305533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc91b-7cd4-48af-b4e4-f2aa23be0c4e service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:57.307515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-79ecc91b-7cd4-48af-b4e4-f2aa23be0c4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:57.326841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc91b-7cd4-48af-b4e4-f2aa23be0c4e service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:57.331851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:57.332992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.333280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.333558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] PATH_INFO: `/v3/users/a47d0a4c80564e1c848c4b87fd661f36` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.334917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47d0a4c80564e1c848c4b87fd661f36)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:57.337455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.337915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 878/1746] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:57 2022] POST /identity/v3/projects => generated 369 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:57.341616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3bcbf-2f32-49e5-99af-12b99e9078a8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:57.436462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47d0a4c80564e1c848c4b87fd661f36'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:57.437323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47d0a4c80564e1c848c4b87fd661f36'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:57.457193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47d0a4c80564e1c848c4b87fd661f36'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:57.458107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47d0a4c80564e1c848c4b87fd661f36'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:57.477597 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3bcbf-2f32-49e5-99af-12b99e9078a8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:57.488309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96184472-2f67-4462-bdfd-b54afcbd86ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:57.489292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96184472-2f67-4462-bdfd-b54afcbd86ec admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.489791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96184472-2f67-4462-bdfd-b54afcbd86ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.490160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96184472-2f67-4462-bdfd-b54afcbd86ec admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.491250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96184472-2f67-4462-bdfd-b54afcbd86ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:57.493490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96184472-2f67-4462-bdfd-b54afcbd86ec admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.495475 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e88cb6e1-b102-4496-bd26-e3a377733ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47d0a4c80564e1c848c4b87fd661f36'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:57.511153 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 869/1747] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:57:57 2022] DELETE /identity/v3/users/a47d0a4c80564e1c848c4b87fd661f36 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:57:57.515357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13faf7-8d76-44b0-acc1-cbbe7abb2f93 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.515649 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13faf7-8d76-44b0-acc1-cbbe7abb2f93 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.515925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13faf7-8d76-44b0-acc1-cbbe7abb2f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.516668 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 879/1748] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:57 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:57:57.520587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96184472-2f67-4462-bdfd-b54afcbd86ec admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:57.523200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-96184472-2f67-4462-bdfd-b54afcbd86ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:57.533528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cc13faf7-8d76-44b0-acc1-cbbe7abb2f93 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:57.535061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cc13faf7-8d76-44b0-acc1-cbbe7abb2f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:57.539697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96184472-2f67-4462-bdfd-b54afcbd86ec admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:57.571977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:57.583039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:57.621631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 870/1749] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:57.631448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13faf7-8d76-44b0-acc1-cbbe7abb2f93 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:57.632916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cc13faf7-8d76-44b0-acc1-cbbe7abb2f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:57.634260 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40df5b-6c51-4db9-ba13-5ed49de2d42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:57.634956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40df5b-6c51-4db9-ba13-5ed49de2d42e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.635287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40df5b-6c51-4db9-ba13-5ed49de2d42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.635561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40df5b-6c51-4db9-ba13-5ed49de2d42e admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.636363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40df5b-6c51-4db9-ba13-5ed49de2d42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:57.638120 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40df5b-6c51-4db9-ba13-5ed49de2d42e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.653764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 880/1750] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:57.654848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40df5b-6c51-4db9-ba13-5ed49de2d42e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:57.666423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40df5b-6c51-4db9-ba13-5ed49de2d42e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:57.672017 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553af3f-2721-430a-8244-54a7cf5e4fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:57.672801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553af3f-2721-430a-8244-54a7cf5e4fad admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.673154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553af3f-2721-430a-8244-54a7cf5e4fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.673430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553af3f-2721-430a-8244-54a7cf5e4fad admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.674165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553af3f-2721-430a-8244-54a7cf5e4fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:57.675829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553af3f-2721-430a-8244-54a7cf5e4fad admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.687621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 881/1751] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:57:57.692127 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553af3f-2721-430a-8244-54a7cf5e4fad admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:57.704665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553af3f-2721-430a-8244-54a7cf5e4fad admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:57.710521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97ec8a-c872-4e38-97dd-77bacc77d68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:57.710704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13faf7-8d76-44b0-acc1-cbbe7abb2f93 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:57.711614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97ec8a-c872-4e38-97dd-77bacc77d68f admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.712285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97ec8a-c872-4e38-97dd-77bacc77d68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.712285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97ec8a-c872-4e38-97dd-77bacc77d68f admin admin] PATH_INFO: `/v3/projects/8031808c6eee4f6fb0ffaefa256d5316/users/1b17191d612d46f895bb13c018ef20a2/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.715345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdd53a-db76-4342-99cd-0343c5f942bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:57.716400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdd53a-db76-4342-99cd-0343c5f942bd service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.716757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdd53a-db76-4342-99cd-0343c5f942bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.717178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdd53a-db76-4342-99cd-0343c5f942bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.733170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97ec8a-c872-4e38-97dd-77bacc77d68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8031808c6eee4f6fb0ffaefa256d5316, user_id=1b17191d612d46f895bb13c018ef20a2, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:57.735696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdd53a-db76-4342-99cd-0343c5f942bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:57.736395 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97ec8a-c872-4e38-97dd-77bacc77d68f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.737276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdd53a-db76-4342-99cd-0343c5f942bd service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.748553 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 871/1752] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:57:57.754405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 882/1753] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:57 2022] PUT /identity/v3/projects/8031808c6eee4f6fb0ffaefa256d5316/users/1b17191d612d46f895bb13c018ef20a2/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:57.764876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d513e0a-d424-4abb-8097-8d9db3ff172b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.765172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d513e0a-d424-4abb-8097-8d9db3ff172b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.765444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d513e0a-d424-4abb-8097-8d9db3ff172b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.793108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6d513e0a-d424-4abb-8097-8d9db3ff172b None None] MFA Rules not processed for user `1b17191d612d46f895bb13c018ef20a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:57.794457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6d513e0a-d424-4abb-8097-8d9db3ff172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:57.875014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 872/1754] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:57.879630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97ec8a-c872-4e38-97dd-77bacc77d68f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:57.952705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97ec8a-c872-4e38-97dd-77bacc77d68f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:57.958181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2573ba-29a0-4eab-90d0-8ac7029b7300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:57.959290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2573ba-29a0-4eab-90d0-8ac7029b7300 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:57.959578 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2573ba-29a0-4eab-90d0-8ac7029b7300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:57.959854 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2573ba-29a0-4eab-90d0-8ac7029b7300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:57.960783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2573ba-29a0-4eab-90d0-8ac7029b7300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:57.965026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2573ba-29a0-4eab-90d0-8ac7029b7300 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:57.995617 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 883/1755] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:57 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:57.999773 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d513e0a-d424-4abb-8097-8d9db3ff172b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:58.017838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d513e0a-d424-4abb-8097-8d9db3ff172b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:58.057955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2573ba-29a0-4eab-90d0-8ac7029b7300 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:58.060383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0a2573ba-29a0-4eab-90d0-8ac7029b7300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:58.095026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1e5de-a347-43c6-8a9f-06f0a3b4bd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:58.095706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1e5de-a347-43c6-8a9f-06f0a3b4bd4a admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.096012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1e5de-a347-43c6-8a9f-06f0a3b4bd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.096424 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1e5de-a347-43c6-8a9f-06f0a3b4bd4a admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.097353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1e5de-a347-43c6-8a9f-06f0a3b4bd4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:58.099583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1e5de-a347-43c6-8a9f-06f0a3b4bd4a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:58.121261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 873/1756] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:57 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:58.125224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1e5de-a347-43c6-8a9f-06f0a3b4bd4a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:58.131686 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:58.135437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1e5de-a347-43c6-8a9f-06f0a3b4bd4a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:58.141476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320df07-bbc6-4d98-931e-52b31a5be8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:58.142209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320df07-bbc6-4d98-931e-52b31a5be8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.142555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320df07-bbc6-4d98-931e-52b31a5be8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.142898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320df07-bbc6-4d98-931e-52b31a5be8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.143734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320df07-bbc6-4d98-931e-52b31a5be8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:58.145445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320df07-bbc6-4d98-931e-52b31a5be8e5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:58.148044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2573ba-29a0-4eab-90d0-8ac7029b7300 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:58.152770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:58.153649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.153931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.154208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] PATH_INFO: `/v3/projects/842dfb5005414bb8a36a395189a80d26` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.155399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=842dfb5005414bb8a36a395189a80d26)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:58.155916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 874/1757] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:57:58.157459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:58.160327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320df07-bbc6-4d98-931e-52b31a5be8e5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:58.170253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320df07-bbc6-4d98-931e-52b31a5be8e5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:58.174040 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:58.174874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f74bb9-2bf3-41b5-9518-01578fc6b65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:58.175951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f74bb9-2bf3-41b5-9518-01578fc6b65c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.176361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f74bb9-2bf3-41b5-9518-01578fc6b65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.176782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f74bb9-2bf3-41b5-9518-01578fc6b65c admin admin] PATH_INFO: `/v3/projects/d8934ce3b83d4925b77ff7dc0b516ddc/users/fff12d0dee1649ed9bd3426c131c450f/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.193509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f74bb9-2bf3-41b5-9518-01578fc6b65c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8934ce3b83d4925b77ff7dc0b516ddc, user_id=fff12d0dee1649ed9bd3426c131c450f, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:58.196299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f74bb9-2bf3-41b5-9518-01578fc6b65c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:58.209827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 875/1758] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:58 2022] PUT /identity/v3/projects/d8934ce3b83d4925b77ff7dc0b516ddc/users/fff12d0dee1649ed9bd3426c131c450f/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:57:58.211449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '842dfb5005414bb8a36a395189a80d26', 'request_id': 'req-f817931c-9e63-4c8d-8ce0-ebcaff18d001'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.214503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f74bb9-2bf3-41b5-9518-01578fc6b65c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:58.218159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '842dfb5005414bb8a36a395189a80d26', 'request_id': 'req-f817931c-9e63-4c8d-8ce0-ebcaff18d001'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.226852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '842dfb5005414bb8a36a395189a80d26', 'request_id': 'req-f817931c-9e63-4c8d-8ce0-ebcaff18d001'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.237561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] The token cache is being invalidate because project 842dfb5005414bb8a36a395189a80d26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:57:58.237871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.238538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.240268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 884/1759] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:57:58 2022] DELETE /identity/v3/projects/842dfb5005414bb8a36a395189a80d26 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:58.244199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:58.245607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:58.276618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f74bb9-2bf3-41b5-9518-01578fc6b65c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:58.282214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18c1ad-db79-4b9f-aa48-15f89089d266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:58.282905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18c1ad-db79-4b9f-aa48-15f89089d266 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.283187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18c1ad-db79-4b9f-aa48-15f89089d266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.283455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18c1ad-db79-4b9f-aa48-15f89089d266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.284316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18c1ad-db79-4b9f-aa48-15f89089d266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:58.285939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18c1ad-db79-4b9f-aa48-15f89089d266 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:58.296393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 876/1760] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:58.300685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18c1ad-db79-4b9f-aa48-15f89089d266 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:58.302606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9e18c1ad-db79-4b9f-aa48-15f89089d266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:58.310420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817931c-9e63-4c8d-8ce0-ebcaff18d001 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:58.315115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:58.316163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18c1ad-db79-4b9f-aa48-15f89089d266 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:58.316450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.316735 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.317012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] PATH_INFO: `/v3/users/147706b62e1d4e64a9ff53cdd32a3059` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.318253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=147706b62e1d4e64a9ff53cdd32a3059)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:58.320231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:58.321375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f82dd-8259-49cd-9859-3472d017ed3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:58.322266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f82dd-8259-49cd-9859-3472d017ed3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.322550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f82dd-8259-49cd-9859-3472d017ed3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.322826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f82dd-8259-49cd-9859-3472d017ed3a admin admin] PATH_INFO: `/v3/projects/d8934ce3b83d4925b77ff7dc0b516ddc/users/fff12d0dee1649ed9bd3426c131c450f/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.325152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f82dd-8259-49cd-9859-3472d017ed3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8934ce3b83d4925b77ff7dc0b516ddc, user_id=fff12d0dee1649ed9bd3426c131c450f, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:58.327996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f82dd-8259-49cd-9859-3472d017ed3a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:58.339654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 877/1761] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:58 2022] PUT /identity/v3/projects/d8934ce3b83d4925b77ff7dc0b516ddc/users/fff12d0dee1649ed9bd3426c131c450f/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:57:58.350695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55011-40b8-4b05-9b37-c36c8f63e9fc None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.350984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55011-40b8-4b05-9b37-c36c8f63e9fc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.351261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55011-40b8-4b05-9b37-c36c8f63e9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.369798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '147706b62e1d4e64a9ff53cdd32a3059'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.378250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e5c55011-40b8-4b05-9b37-c36c8f63e9fc None None] MFA Rules not processed for user `fff12d0dee1649ed9bd3426c131c450f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:58.378415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '147706b62e1d4e64a9ff53cdd32a3059'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.379041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '147706b62e1d4e64a9ff53cdd32a3059'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.379598 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e5c55011-40b8-4b05-9b37-c36c8f63e9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:58.387525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '147706b62e1d4e64a9ff53cdd32a3059'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.399613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '147706b62e1d4e64a9ff53cdd32a3059'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.406139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 885/1762] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:57:58 2022] DELETE /identity/v3/users/147706b62e1d4e64a9ff53cdd32a3059 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:57:58.451053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 878/1763] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:58 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:58.463190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:58.465056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:58.481084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55011-40b8-4b05-9b37-c36c8f63e9fc None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:58.482803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e5c55011-40b8-4b05-9b37-c36c8f63e9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:58.517698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:58.534674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:58.560024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55011-40b8-4b05-9b37-c36c8f63e9fc None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:58.575828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76b606-41de-4930-9c98-369f84002bb8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:58.586045 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d9c3e-0bb9-4ebf-ac9f-4d92e58937b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:58.587102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d9c3e-0bb9-4ebf-ac9f-4d92e58937b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.587977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d9c3e-0bb9-4ebf-ac9f-4d92e58937b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.588427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d9c3e-0bb9-4ebf-ac9f-4d92e58937b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.653144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:58.654657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.654943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.655217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] PATH_INFO: `/v3/users/5e0dc3493bcf47be966e77d6c43b7fff` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.656951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e0dc3493bcf47be966e77d6c43b7fff)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:58.659018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:58.674412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d9c3e-0bb9-4ebf-ac9f-4d92e58937b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:58.676624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d9c3e-0bb9-4ebf-ac9f-4d92e58937b9 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:58.696310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 886/1764] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:58 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:57:58.736001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e0dc3493bcf47be966e77d6c43b7fff'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.737607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e0dc3493bcf47be966e77d6c43b7fff'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.747207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e0dc3493bcf47be966e77d6c43b7fff'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.747948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e0dc3493bcf47be966e77d6c43b7fff'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.765252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e0dc3493bcf47be966e77d6c43b7fff'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:58.785753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 879/1765] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:57:58 2022] DELETE /identity/v3/users/5e0dc3493bcf47be966e77d6c43b7fff => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:57:58.790853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe07124-7e7b-43d9-95ac-79d43bffdb81 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.791180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe07124-7e7b-43d9-95ac-79d43bffdb81 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.791454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe07124-7e7b-43d9-95ac-79d43bffdb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.799493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:58.801612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:58.836556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cbe07124-7e7b-43d9-95ac-79d43bffdb81 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:58.841259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cbe07124-7e7b-43d9-95ac-79d43bffdb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:58.922621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab542631-dd27-4ec8-bc07-1a4ac14a2234 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:58.933410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:58.934510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:58.935020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:58.935270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] PATH_INFO: `/v3/projects/b523284a58444be89d5bc58cc2697a76` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:58.936599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b523284a58444be89d5bc58cc2697a76)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:58.938526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:58.945543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:58.984627 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 887/1766] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:58.995956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe07124-7e7b-43d9-95ac-79d43bffdb81 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.000938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cbe07124-7e7b-43d9-95ac-79d43bffdb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:59.018740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b523284a58444be89d5bc58cc2697a76', 'request_id': 'req-d1a16c9c-e7d7-406f-8324-a98895334dbc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:59.025543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b523284a58444be89d5bc58cc2697a76', 'request_id': 'req-d1a16c9c-e7d7-406f-8324-a98895334dbc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:59.036194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b523284a58444be89d5bc58cc2697a76', 'request_id': 'req-d1a16c9c-e7d7-406f-8324-a98895334dbc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:59.052994 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] The token cache is being invalidate because project b523284a58444be89d5bc58cc2697a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:57:59.053286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:59.053979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d1a16c9c-e7d7-406f-8324-a98895334dbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:59.060160 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 880/1767] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:57:58 2022] DELETE /identity/v3/projects/b523284a58444be89d5bc58cc2697a76 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:59.061969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e903e-e5c0-4fd9-becb-70425154bf6a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.062258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e903e-e5c0-4fd9-becb-70425154bf6a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.062580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e903e-e5c0-4fd9-becb-70425154bf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.078319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe07124-7e7b-43d9-95ac-79d43bffdb81 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:59.091711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9a7e903e-e5c0-4fd9-becb-70425154bf6a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:59.093407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9a7e903e-e5c0-4fd9-becb-70425154bf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:59.130575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:59.158852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136879c5-f181-44ad-b1ec-7e466af1b84b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:59.159623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136879c5-f181-44ad-b1ec-7e466af1b84b service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.159905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136879c5-f181-44ad-b1ec-7e466af1b84b service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.160207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136879c5-f181-44ad-b1ec-7e466af1b84b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.162412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-136879c5-f181-44ad-b1ec-7e466af1b84b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:59.172161 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 881/1768] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:57:59.184882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e189238-ce7c-4587-8754-b2d6b2598268 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.185177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e189238-ce7c-4587-8754-b2d6b2598268 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.185460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e189238-ce7c-4587-8754-b2d6b2598268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.188765 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136879c5-f181-44ad-b1ec-7e466af1b84b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:59.190389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136879c5-f181-44ad-b1ec-7e466af1b84b service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:59.205886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 888/1769] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:57:59.207292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7e189238-ce7c-4587-8754-b2d6b2598268 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:59.208567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7e189238-ce7c-4587-8754-b2d6b2598268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:59.217119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 882/1770] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:59.223663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136879c5-f181-44ad-b1ec-7e466af1b84b service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.234008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136879c5-f181-44ad-b1ec-7e466af1b84b service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:59.239880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e59e14-bd2d-4b30-b2f2-be52bee5aa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:59.240753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e59e14-bd2d-4b30-b2f2-be52bee5aa3f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.240873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e59e14-bd2d-4b30-b2f2-be52bee5aa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.241119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e59e14-bd2d-4b30-b2f2-be52bee5aa3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.241858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e59e14-bd2d-4b30-b2f2-be52bee5aa3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:59.242166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e59e14-bd2d-4b30-b2f2-be52bee5aa3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:59.243840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e59e14-bd2d-4b30-b2f2-be52bee5aa3f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:59.259913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 889/1771] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:57:59.261949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e189238-ce7c-4587-8754-b2d6b2598268 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.272036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e189238-ce7c-4587-8754-b2d6b2598268 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:59.278515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f16e3-466a-4541-bb09-c59b146ced66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:59.279200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f16e3-466a-4541-bb09-c59b146ced66 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.279480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f16e3-466a-4541-bb09-c59b146ced66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.279893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f16e3-466a-4541-bb09-c59b146ced66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.280486 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f16e3-466a-4541-bb09-c59b146ced66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:59.285283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f16e3-466a-4541-bb09-c59b146ced66 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:59.323926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 883/1772] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:59 2022] POST /identity/v3/projects => generated 399 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:57:59.325097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e59e14-bd2d-4b30-b2f2-be52bee5aa3f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.336940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e59e14-bd2d-4b30-b2f2-be52bee5aa3f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:59.412488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46642e82-46e5-49ab-9bc3-d8a250ddf984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:59.413311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46642e82-46e5-49ab-9bc3-d8a250ddf984 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.413785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46642e82-46e5-49ab-9bc3-d8a250ddf984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.414055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46642e82-46e5-49ab-9bc3-d8a250ddf984 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.414949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46642e82-46e5-49ab-9bc3-d8a250ddf984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:59.417063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46642e82-46e5-49ab-9bc3-d8a250ddf984 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:59.444341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 890/1773] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:59 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:57:59.447860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f16e3-466a-4541-bb09-c59b146ced66 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.460305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f16e3-466a-4541-bb09-c59b146ced66 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:59.466527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60151b1b-9a85-47cf-b7cf-312973dad6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:59.467419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60151b1b-9a85-47cf-b7cf-312973dad6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.467869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60151b1b-9a85-47cf-b7cf-312973dad6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.468447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60151b1b-9a85-47cf-b7cf-312973dad6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.469334 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60151b1b-9a85-47cf-b7cf-312973dad6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:59.472379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60151b1b-9a85-47cf-b7cf-312973dad6d0 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:59.484084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 884/1774] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:59.488777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46642e82-46e5-49ab-9bc3-d8a250ddf984 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.499777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46642e82-46e5-49ab-9bc3-d8a250ddf984 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:59.511962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db406145-c64c-4396-8a29-bb7400d363c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:59.512901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db406145-c64c-4396-8a29-bb7400d363c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.513235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db406145-c64c-4396-8a29-bb7400d363c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.513538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db406145-c64c-4396-8a29-bb7400d363c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.514428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db406145-c64c-4396-8a29-bb7400d363c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:59.516070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db406145-c64c-4396-8a29-bb7400d363c9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:59.530612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 891/1775] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:57:59.535465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60151b1b-9a85-47cf-b7cf-312973dad6d0 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.561902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60151b1b-9a85-47cf-b7cf-312973dad6d0 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:59.572134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db406145-c64c-4396-8a29-bb7400d363c9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.573922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-db406145-c64c-4396-8a29-bb7400d363c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:59.578914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a1053-8908-437b-94c6-f76aa12ebf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:59.580196 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a1053-8908-437b-94c6-f76aa12ebf5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.580661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a1053-8908-437b-94c6-f76aa12ebf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.581083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a1053-8908-437b-94c6-f76aa12ebf5e admin admin] PATH_INFO: `/v3/projects/c8ebad9e8fac4917bb1d8ec059fe847d/users/fb10576d5c91446183e2e4afcc00218b/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.590109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db406145-c64c-4396-8a29-bb7400d363c9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:59.595567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:59.596759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.597425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.597425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] PATH_INFO: `/v3/projects/3b0d3472514e490184dfa15f661773fc` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.599201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0d3472514e490184dfa15f661773fc)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:59.602099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:59.607210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a1053-8908-437b-94c6-f76aa12ebf5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ebad9e8fac4917bb1d8ec059fe847d, user_id=fb10576d5c91446183e2e4afcc00218b, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:59.609995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a1053-8908-437b-94c6-f76aa12ebf5e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:59.633468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 885/1776] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:57:59 2022] PUT /identity/v3/projects/c8ebad9e8fac4917bb1d8ec059fe847d/users/fb10576d5c91446183e2e4afcc00218b/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:57:59.643497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dda739-ea56-419b-ad84-d3b14e72cce6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.643982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dda739-ea56-419b-ad84-d3b14e72cce6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.644320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dda739-ea56-419b-ad84-d3b14e72cce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.670246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0d3472514e490184dfa15f661773fc', 'request_id': 'req-6b3e7ee0-3c41-444f-9f60-c75dffff90da'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:59.677346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-79dda739-ea56-419b-ad84-d3b14e72cce6 None None] MFA Rules not processed for user `fb10576d5c91446183e2e4afcc00218b`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:59.678733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-79dda739-ea56-419b-ad84-d3b14e72cce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:59.679287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0d3472514e490184dfa15f661773fc', 'request_id': 'req-6b3e7ee0-3c41-444f-9f60-c75dffff90da'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:59.692942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0d3472514e490184dfa15f661773fc', 'request_id': 'req-6b3e7ee0-3c41-444f-9f60-c75dffff90da'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:59.702071 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:59.709747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] The token cache is being invalidate because project 3b0d3472514e490184dfa15f661773fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:57:59.710135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:59.710882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b3e7ee0-3c41-444f-9f60-c75dffff90da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:57:59.713006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 892/1777] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:57:59 2022] DELETE /identity/v3/projects/3b0d3472514e490184dfa15f661773fc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:57:59.717841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74372f59-0ea7-4e98-b2e1-b1b416b83907 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.718132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74372f59-0ea7-4e98-b2e1-b1b416b83907 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.718561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74372f59-0ea7-4e98-b2e1-b1b416b83907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.737107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-74372f59-0ea7-4e98-b2e1-b1b416b83907 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:59.738313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-74372f59-0ea7-4e98-b2e1-b1b416b83907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:59.753471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:57:59.762679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 886/1778] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:59.767156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dda739-ea56-419b-ad84-d3b14e72cce6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.768565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-79dda739-ea56-419b-ad84-d3b14e72cce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:59.838830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 893/1779] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:57:59.849104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e34c87-c26d-4072-bdcb-940b1b23ac7d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.851591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e34c87-c26d-4072-bdcb-940b1b23ac7d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.851914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e34c87-c26d-4072-bdcb-940b1b23ac7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.874133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dda739-ea56-419b-ad84-d3b14e72cce6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:59.877217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f9e34c87-c26d-4072-bdcb-940b1b23ac7d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:57:59.878539 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f9e34c87-c26d-4072-bdcb-940b1b23ac7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:57:59.884805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4fa3c-cd6c-44b9-9b9d-b7f743379bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:59.885892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4fa3c-cd6c-44b9-9b9d-b7f743379bcd admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.886317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4fa3c-cd6c-44b9-9b9d-b7f743379bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.886817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4fa3c-cd6c-44b9-9b9d-b7f743379bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.887803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4fa3c-cd6c-44b9-9b9d-b7f743379bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:59.889967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4fa3c-cd6c-44b9-9b9d-b7f743379bcd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:59.891834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 894/1780] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:57:59.896093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e34c87-c26d-4072-bdcb-940b1b23ac7d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.912350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e34c87-c26d-4072-bdcb-940b1b23ac7d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:57:59.925448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 887/1781] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:59 2022] POST /identity/v3/projects => generated 401 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:57:59.931150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4fa3c-cd6c-44b9-9b9d-b7f743379bcd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.934531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0702cc1-5b94-48a7-95af-d8a71a317082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:57:59.935229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0702cc1-5b94-48a7-95af-d8a71a317082 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:57:59.936023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0702cc1-5b94-48a7-95af-d8a71a317082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:57:59.936545 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0702cc1-5b94-48a7-95af-d8a71a317082 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:57:59.937377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0702cc1-5b94-48a7-95af-d8a71a317082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:57:59.937695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0702cc1-5b94-48a7-95af-d8a71a317082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:57:59.941811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0702cc1-5b94-48a7-95af-d8a71a317082 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:57:59.972148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 895/1782] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:57:59.977276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0702cc1-5b94-48a7-95af-d8a71a317082 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:57:59.992754 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0702cc1-5b94-48a7-95af-d8a71a317082 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.032559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4fa3c-cd6c-44b9-9b9d-b7f743379bcd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.045575 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb90916-38e5-4154-882d-b8f372491068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.046277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb90916-38e5-4154-882d-b8f372491068 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.046567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb90916-38e5-4154-882d-b8f372491068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.046845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb90916-38e5-4154-882d-b8f372491068 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.047556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb90916-38e5-4154-882d-b8f372491068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.050300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb90916-38e5-4154-882d-b8f372491068 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.077292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 888/1783] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:57:59 2022] POST /identity/v3/users => generated 307 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:00.082873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb90916-38e5-4154-882d-b8f372491068 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.095214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb90916-38e5-4154-882d-b8f372491068 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.101134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499fae4-08c4-40b1-8ed9-1a9399d021f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.101984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fae4-08c4-40b1-8ed9-1a9399d021f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.102414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fae4-08c4-40b1-8ed9-1a9399d021f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.102823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fae4-08c4-40b1-8ed9-1a9399d021f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.103240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ed97b-df77-4060-b53d-2db26409ca55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.103864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ed97b-df77-4060-b53d-2db26409ca55 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.104502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ed97b-df77-4060-b53d-2db26409ca55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.104825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ed97b-df77-4060-b53d-2db26409ca55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.105012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6499fae4-08c4-40b1-8ed9-1a9399d021f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.105859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ed97b-df77-4060-b53d-2db26409ca55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.107868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6499fae4-08c4-40b1-8ed9-1a9399d021f5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.110236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ed97b-df77-4060-b53d-2db26409ca55 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.137617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 889/1784] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:00.138571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499fae4-08c4-40b1-8ed9-1a9399d021f5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.152064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499fae4-08c4-40b1-8ed9-1a9399d021f5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.156386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 896/1785] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:57:59 2022] POST /identity/v3/projects => generated 405 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:00.158175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ed97b-df77-4060-b53d-2db26409ca55 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.184572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:00.246812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ed97b-df77-4060-b53d-2db26409ca55 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.247063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e603dc-5bce-4871-a0b9-ddb593fd8634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.248152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e603dc-5bce-4871-a0b9-ddb593fd8634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.248152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e603dc-5bce-4871-a0b9-ddb593fd8634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.248334 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e603dc-5bce-4871-a0b9-ddb593fd8634 admin admin] PATH_INFO: `/v3/projects/f4392c139639424a85a9ec046495ac11/users/e46ec4c07a44456f8a23e7a918757e2c/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.253565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2304e-4d40-4faf-ac6f-139fc3ab8e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.254490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2304e-4d40-4faf-ac6f-139fc3ab8e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.254886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2304e-4d40-4faf-ac6f-139fc3ab8e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.255267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2304e-4d40-4faf-ac6f-139fc3ab8e56 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.256319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2304e-4d40-4faf-ac6f-139fc3ab8e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.259429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2304e-4d40-4faf-ac6f-139fc3ab8e56 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.276391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e603dc-5bce-4871-a0b9-ddb593fd8634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4392c139639424a85a9ec046495ac11, user_id=e46ec4c07a44456f8a23e7a918757e2c, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.282937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e603dc-5bce-4871-a0b9-ddb593fd8634 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.300636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 897/1786] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:00 2022] POST /identity/v3/users => generated 303 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:00.301431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2304e-4d40-4faf-ac6f-139fc3ab8e56 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.303413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 890/1787] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:00 2022] PUT /identity/v3/projects/f4392c139639424a85a9ec046495ac11/users/e46ec4c07a44456f8a23e7a918757e2c/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:00.307861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e603dc-5bce-4871-a0b9-ddb593fd8634 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.313593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2304e-4d40-4faf-ac6f-139fc3ab8e56 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.350090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:00.414544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e603dc-5bce-4871-a0b9-ddb593fd8634 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.417199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ccabc-5c85-485d-8d1f-c950b889e974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.418006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ccabc-5c85-485d-8d1f-c950b889e974 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.418293 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ccabc-5c85-485d-8d1f-c950b889e974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.418616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ccabc-5c85-485d-8d1f-c950b889e974 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.419648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ccabc-5c85-485d-8d1f-c950b889e974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.421653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ccabc-5c85-485d-8d1f-c950b889e974 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.423231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000c34d-1532-4018-9c26-1ac088fa2588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.424248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000c34d-1532-4018-9c26-1ac088fa2588 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.424536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000c34d-1532-4018-9c26-1ac088fa2588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.424846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000c34d-1532-4018-9c26-1ac088fa2588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.425740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000c34d-1532-4018-9c26-1ac088fa2588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.428341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000c34d-1532-4018-9c26-1ac088fa2588 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.436488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 898/1788] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:00.440542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ccabc-5c85-485d-8d1f-c950b889e974 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.443219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 891/1789] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:00.447108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000c34d-1532-4018-9c26-1ac088fa2588 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.452566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ccabc-5c85-485d-8d1f-c950b889e974 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.460047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cd9f2-9b0e-41f3-bc24-07eda31e81ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.460984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cd9f2-9b0e-41f3-bc24-07eda31e81ac admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.461441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cd9f2-9b0e-41f3-bc24-07eda31e81ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.461554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000c34d-1532-4018-9c26-1ac088fa2588 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.461978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cd9f2-9b0e-41f3-bc24-07eda31e81ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.462871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cd9f2-9b0e-41f3-bc24-07eda31e81ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.464790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cd9f2-9b0e-41f3-bc24-07eda31e81ac admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.468801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6fbc8-6760-430a-877f-dce7359e189c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.469702 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6fbc8-6760-430a-877f-dce7359e189c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.469981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6fbc8-6760-430a-877f-dce7359e189c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.470265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6fbc8-6760-430a-877f-dce7359e189c admin admin] PATH_INFO: `/v3/projects/f4392c139639424a85a9ec046495ac11/users/e46ec4c07a44456f8a23e7a918757e2c/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.475800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6fbc8-6760-430a-877f-dce7359e189c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4392c139639424a85a9ec046495ac11, user_id=e46ec4c07a44456f8a23e7a918757e2c, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.478531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6fbc8-6760-430a-877f-dce7359e189c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.480333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 899/1790] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:00.484534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cd9f2-9b0e-41f3-bc24-07eda31e81ac admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.492930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 892/1791] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:00 2022] PUT /identity/v3/projects/f4392c139639424a85a9ec046495ac11/users/e46ec4c07a44456f8a23e7a918757e2c/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:00.500568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cd9f2-9b0e-41f3-bc24-07eda31e81ac admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.503137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9ffcb-3693-4122-bdb1-432f37c09988 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.503590 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9ffcb-3693-4122-bdb1-432f37c09988 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.504136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9ffcb-3693-4122-bdb1-432f37c09988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.541735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-53a9ffcb-3693-4122-bdb1-432f37c09988 None None] MFA Rules not processed for user `e46ec4c07a44456f8a23e7a918757e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:00.543078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-53a9ffcb-3693-4122-bdb1-432f37c09988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:00.571762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fb964-8c80-4532-bb38-4247b421a780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.572720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fb964-8c80-4532-bb38-4247b421a780 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.573004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fb964-8c80-4532-bb38-4247b421a780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.573284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fb964-8c80-4532-bb38-4247b421a780 admin admin] PATH_INFO: `/v3/projects/3c887532702d4c20baddf1f710c5a762/users/32b848a140014907b1db2e75232ac7b3/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.594666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fb964-8c80-4532-bb38-4247b421a780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c887532702d4c20baddf1f710c5a762, user_id=32b848a140014907b1db2e75232ac7b3, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.597736 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fb964-8c80-4532-bb38-4247b421a780 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.611304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 900/1792] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:00 2022] PUT /identity/v3/projects/3c887532702d4c20baddf1f710c5a762/users/32b848a140014907b1db2e75232ac7b3/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:00.617196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fb964-8c80-4532-bb38-4247b421a780 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.619782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e48fb964-8c80-4532-bb38-4247b421a780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:00.627338 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 893/1793] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:00.628960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cd60b-a483-409c-a4a4-25e3e13f5f94 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.629369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cd60b-a483-409c-a4a4-25e3e13f5f94 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.629752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cd60b-a483-409c-a4a4-25e3e13f5f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.658086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-916cd60b-a483-409c-a4a4-25e3e13f5f94 None None] MFA Rules not processed for user `32b848a140014907b1db2e75232ac7b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:00.659320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-916cd60b-a483-409c-a4a4-25e3e13f5f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:00.663326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:00.686616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fb964-8c80-4532-bb38-4247b421a780 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.691839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.692764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.693040 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.693312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] PATH_INFO: `/v3/users/1b17191d612d46f895bb13c018ef20a2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.694521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b17191d612d46f895bb13c018ef20a2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.696501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.731073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 894/1794] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:00.731073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916cd60b-a483-409c-a4a4-25e3e13f5f94 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.732898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-916cd60b-a483-409c-a4a4-25e3e13f5f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:00.732987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:00.755314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b17191d612d46f895bb13c018ef20a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:00.763512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b17191d612d46f895bb13c018ef20a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:00.764328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b17191d612d46f895bb13c018ef20a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:00.773230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b17191d612d46f895bb13c018ef20a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:00.785923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b17191d612d46f895bb13c018ef20a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:00.788242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916cd60b-a483-409c-a4a4-25e3e13f5f94 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.791752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541c9a1-4a5c-4d7d-a524-feee59fd33e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.792128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 901/1795] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:00 2022] DELETE /identity/v3/users/1b17191d612d46f895bb13c018ef20a2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:00.792435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541c9a1-4a5c-4d7d-a524-feee59fd33e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.792714 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541c9a1-4a5c-4d7d-a524-feee59fd33e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.792986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541c9a1-4a5c-4d7d-a524-feee59fd33e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.793106 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.794453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:00.794698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1541c9a1-4a5c-4d7d-a524-feee59fd33e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:00.859905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541c9a1-4a5c-4d7d-a524-feee59fd33e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.861390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541c9a1-4a5c-4d7d-a524-feee59fd33e3 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.864188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf719d9-6660-4f62-add5-51d2bf3bc412 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.869026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fcb98-5c42-47b8-a137-826ed5575dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.869727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fcb98-5c42-47b8-a137-826ed5575dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.870011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fcb98-5c42-47b8-a137-826ed5575dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.870290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fcb98-5c42-47b8-a137-826ed5575dd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.870986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fcb98-5c42-47b8-a137-826ed5575dd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.871383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 895/1796] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:00 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:00.872682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549d264-b6d7-485b-b852-21fbd85bb740 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.872971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549d264-b6d7-485b-b852-21fbd85bb740 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.873163 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fcb98-5c42-47b8-a137-826ed5575dd5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.873266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549d264-b6d7-485b-b852-21fbd85bb740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.891560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1549d264-b6d7-485b-b852-21fbd85bb740 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:00.892991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1549d264-b6d7-485b-b852-21fbd85bb740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:00.900920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 902/1797] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:00 2022] POST /identity/v3/projects => generated 405 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:00.905700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fcb98-5c42-47b8-a137-826ed5575dd5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.910855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 896/1798] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:00.918055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549d264-b6d7-485b-b852-21fbd85bb740 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:00.919836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1549d264-b6d7-485b-b852-21fbd85bb740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:00.974991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fcb98-5c42-47b8-a137-826ed5575dd5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:00.979864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a8da3-9fb4-4a12-810e-d1970ae823c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:00.980656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a8da3-9fb4-4a12-810e-d1970ae823c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:00.980941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a8da3-9fb4-4a12-810e-d1970ae823c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:00.981276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a8da3-9fb4-4a12-810e-d1970ae823c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:00.981922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a8da3-9fb4-4a12-810e-d1970ae823c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:00.983646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:00.983881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a8da3-9fb4-4a12-810e-d1970ae823c1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:00.987967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549d264-b6d7-485b-b852-21fbd85bb740 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:01.006474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 903/1799] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:00 2022] POST /identity/v3/users => generated 309 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:01.007524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a8da3-9fb4-4a12-810e-d1970ae823c1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:01.010312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4d1a8da3-9fb4-4a12-810e-d1970ae823c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:01.031328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:01.048570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38d2ea-9e2b-4440-8aa8-b950d6195f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:01.049243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38d2ea-9e2b-4440-8aa8-b950d6195f43 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:01.049528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38d2ea-9e2b-4440-8aa8-b950d6195f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:01.049878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38d2ea-9e2b-4440-8aa8-b950d6195f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:01.062101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38d2ea-9e2b-4440-8aa8-b950d6195f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:01.063562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38d2ea-9e2b-4440-8aa8-b950d6195f43 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:01.073294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a8da3-9fb4-4a12-810e-d1970ae823c1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:01.075788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 897/1800] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:01.076817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38d2ea-9e2b-4440-8aa8-b950d6195f43 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:01.081697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7b462-c353-42aa-b85a-9e5919fed133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:01.082486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7b462-c353-42aa-b85a-9e5919fed133 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:01.082777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7b462-c353-42aa-b85a-9e5919fed133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:01.083054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7b462-c353-42aa-b85a-9e5919fed133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:01.086592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38d2ea-9e2b-4440-8aa8-b950d6195f43 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:01.088982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d3e7b462-c353-42aa-b85a-9e5919fed133 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:01.091591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621b7c9-b7e7-42e2-bd09-9c41f9e96053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:01.092841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621b7c9-b7e7-42e2-bd09-9c41f9e96053 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:01.093304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621b7c9-b7e7-42e2-bd09-9c41f9e96053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:01.093668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621b7c9-b7e7-42e2-bd09-9c41f9e96053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:01.094847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621b7c9-b7e7-42e2-bd09-9c41f9e96053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:01.096666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621b7c9-b7e7-42e2-bd09-9c41f9e96053 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:01.111780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 898/1801] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:01.112833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621b7c9-b7e7-42e2-bd09-9c41f9e96053 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:01.127859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621b7c9-b7e7-42e2-bd09-9c41f9e96053 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:01.135772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3ada0-3b89-4c7f-aa14-184e18ae159d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:01.137481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ada0-3b89-4c7f-aa14-184e18ae159d admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:01.137949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ada0-3b89-4c7f-aa14-184e18ae159d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:01.138228 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ada0-3b89-4c7f-aa14-184e18ae159d admin admin] PATH_INFO: `/v3/projects/05a174544d44430d98de09045c03336e/users/2e2fb8d844bb4974aa4a8956763a91dc/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:01.173165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3ada0-3b89-4c7f-aa14-184e18ae159d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a174544d44430d98de09045c03336e, user_id=2e2fb8d844bb4974aa4a8956763a91dc, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:01.176954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3ada0-3b89-4c7f-aa14-184e18ae159d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:01.193851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7b462-c353-42aa-b85a-9e5919fed133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:01.195757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7b462-c353-42aa-b85a-9e5919fed133 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:01.196148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 899/1802] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:01 2022] PUT /identity/v3/projects/05a174544d44430d98de09045c03336e/users/2e2fb8d844bb4974aa4a8956763a91dc/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:01.200357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3ada0-3b89-4c7f-aa14-184e18ae159d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:01.212300 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 904/1803] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:01.278261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3ada0-3b89-4c7f-aa14-184e18ae159d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:01.284279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f514a-09ff-409c-a90a-d24c05b1a1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:01.285214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f514a-09ff-409c-a90a-d24c05b1a1be admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:01.285501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f514a-09ff-409c-a90a-d24c05b1a1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:01.285934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f514a-09ff-409c-a90a-d24c05b1a1be admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:01.286574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f514a-09ff-409c-a90a-d24c05b1a1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:01.288231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f514a-09ff-409c-a90a-d24c05b1a1be admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:01.304840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 900/1804] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:01.306196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7b462-c353-42aa-b85a-9e5919fed133 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:01.320868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7b462-c353-42aa-b85a-9e5919fed133 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:01.326043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b37cb-cdae-4e38-b253-6e5b84cefa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:01.327305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b37cb-cdae-4e38-b253-6e5b84cefa8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:01.327957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b37cb-cdae-4e38-b253-6e5b84cefa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:01.328429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b37cb-cdae-4e38-b253-6e5b84cefa8f admin admin] PATH_INFO: `/v3/projects/05a174544d44430d98de09045c03336e/users/2e2fb8d844bb4974aa4a8956763a91dc/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:01.331056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b37cb-cdae-4e38-b253-6e5b84cefa8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a174544d44430d98de09045c03336e, user_id=2e2fb8d844bb4974aa4a8956763a91dc, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:01.334558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b37cb-cdae-4e38-b253-6e5b84cefa8f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:01.348588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 905/1805] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:01 2022] PUT /identity/v3/projects/05a174544d44430d98de09045c03336e/users/2e2fb8d844bb4974aa4a8956763a91dc/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:01.360042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab625fe9-4af0-4ba5-98aa-14c9fab611e2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:01.361048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab625fe9-4af0-4ba5-98aa-14c9fab611e2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:01.361479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab625fe9-4af0-4ba5-98aa-14c9fab611e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:01.402264 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ab625fe9-4af0-4ba5-98aa-14c9fab611e2 None None] MFA Rules not processed for user `2e2fb8d844bb4974aa4a8956763a91dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:01.403497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ab625fe9-4af0-4ba5-98aa-14c9fab611e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:01.475254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:01.491191 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 901/1806] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:01 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:01.493889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b37cb-cdae-4e38-b253-6e5b84cefa8f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:01.498677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dd7b37cb-cdae-4e38-b253-6e5b84cefa8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:01.502400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab625fe9-4af0-4ba5-98aa-14c9fab611e2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:01.509855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ab625fe9-4af0-4ba5-98aa-14c9fab611e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:01.746407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab625fe9-4af0-4ba5-98aa-14c9fab611e2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:01.750222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1909f6-c587-4106-9d9b-e3165b17c1c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:01.751051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1909f6-c587-4106-9d9b-e3165b17c1c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:01.751338 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1909f6-c587-4106-9d9b-e3165b17c1c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:01.751612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1909f6-c587-4106-9d9b-e3165b17c1c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:01.763834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1909f6-c587-4106-9d9b-e3165b17c1c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:01.765980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1909f6-c587-4106-9d9b-e3165b17c1c2 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:01.768794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b37cb-cdae-4e38-b253-6e5b84cefa8f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:01.778629 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 902/1807] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:01 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:01.792146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:01.793076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:01.793343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:01.793618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] PATH_INFO: `/v3/projects/8031808c6eee4f6fb0ffaefa256d5316` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:01.798765 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8031808c6eee4f6fb0ffaefa256d5316)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:01.802734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:01.908429 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8031808c6eee4f6fb0ffaefa256d5316', 'request_id': 'req-affd0d06-a12c-427c-be2a-a2072dee40cd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:01.916287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8031808c6eee4f6fb0ffaefa256d5316', 'request_id': 'req-affd0d06-a12c-427c-be2a-a2072dee40cd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:01.926027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8031808c6eee4f6fb0ffaefa256d5316', 'request_id': 'req-affd0d06-a12c-427c-be2a-a2072dee40cd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:01.929428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:01.943154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] The token cache is being invalidate because project 8031808c6eee4f6fb0ffaefa256d5316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:01.943616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:01.944563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:01.946289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 906/1808] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:01 2022] DELETE /identity/v3/projects/8031808c6eee4f6fb0ffaefa256d5316 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:01.950252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1909f6-c587-4106-9d9b-e3165b17c1c2 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:01.962117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1909f6-c587-4106-9d9b-e3165b17c1c2 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:02.031078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacc163-4177-4626-9494-650875624587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:02.032137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacc163-4177-4626-9494-650875624587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:02.032544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacc163-4177-4626-9494-650875624587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:02.032881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacc163-4177-4626-9494-650875624587 admin admin] PATH_INFO: `/v3/users/fff12d0dee1649ed9bd3426c131c450f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:02.034599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacc163-4177-4626-9494-650875624587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fff12d0dee1649ed9bd3426c131c450f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:02.036694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacc163-4177-4626-9494-650875624587 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:02.094597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8eacc163-4177-4626-9494-650875624587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fff12d0dee1649ed9bd3426c131c450f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:02.095272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8eacc163-4177-4626-9494-650875624587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fff12d0dee1649ed9bd3426c131c450f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:02.104081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8eacc163-4177-4626-9494-650875624587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fff12d0dee1649ed9bd3426c131c450f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:02.104807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8eacc163-4177-4626-9494-650875624587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fff12d0dee1649ed9bd3426c131c450f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:02.118613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8eacc163-4177-4626-9494-650875624587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fff12d0dee1649ed9bd3426c131c450f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:02.136094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 903/1809] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:01 2022] DELETE /identity/v3/users/fff12d0dee1649ed9bd3426c131c450f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:02.542448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:02.543973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:02.557289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd0d06-a12c-427c-be2a-a2072dee40cd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:02.633023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:02.634018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:02.634363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:02.634709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] PATH_INFO: `/v3/projects/d8934ce3b83d4925b77ff7dc0b516ddc` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:02.635982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8934ce3b83d4925b77ff7dc0b516ddc)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:02.638053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:02.689833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8934ce3b83d4925b77ff7dc0b516ddc', 'request_id': 'req-4996bd59-5caa-48df-8393-1aa7996b99c7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:02.696786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8934ce3b83d4925b77ff7dc0b516ddc', 'request_id': 'req-4996bd59-5caa-48df-8393-1aa7996b99c7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:02.706349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8934ce3b83d4925b77ff7dc0b516ddc', 'request_id': 'req-4996bd59-5caa-48df-8393-1aa7996b99c7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:02.717988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] The token cache is being invalidate because project d8934ce3b83d4925b77ff7dc0b516ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:02.718287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:02.719002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4996bd59-5caa-48df-8393-1aa7996b99c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:02.720739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 907/1810] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:02 2022] DELETE /identity/v3/projects/d8934ce3b83d4925b77ff7dc0b516ddc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:02.726339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e854a3-1b8d-4bb9-9825-10901bf35471 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:02.726814 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e854a3-1b8d-4bb9-9825-10901bf35471 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:02.726900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e854a3-1b8d-4bb9-9825-10901bf35471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:02.748311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-98e854a3-1b8d-4bb9-9825-10901bf35471 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:02.749645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-98e854a3-1b8d-4bb9-9825-10901bf35471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:02.761211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:02.822592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 904/1811] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:02.831097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703376d-6dc0-457e-9847-a682fa9b4fc6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:02.831586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703376d-6dc0-457e-9847-a682fa9b4fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:02.831913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703376d-6dc0-457e-9847-a682fa9b4fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:02.855996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b703376d-6dc0-457e-9847-a682fa9b4fc6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:02.857238 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b703376d-6dc0-457e-9847-a682fa9b4fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:02.865846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 908/1812] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:02.871773 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e854a3-1b8d-4bb9-9825-10901bf35471 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:02.884416 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e854a3-1b8d-4bb9-9825-10901bf35471 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:02.890171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df1d67-619a-4b36-8bdd-4b88681a12b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:02.890869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df1d67-619a-4b36-8bdd-4b88681a12b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:02.891147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df1d67-619a-4b36-8bdd-4b88681a12b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:02.891447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df1d67-619a-4b36-8bdd-4b88681a12b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:02.892192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df1d67-619a-4b36-8bdd-4b88681a12b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:02.892498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df1d67-619a-4b36-8bdd-4b88681a12b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:02.894331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df1d67-619a-4b36-8bdd-4b88681a12b2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:02.909471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 905/1813] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:02.914410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703376d-6dc0-457e-9847-a682fa9b4fc6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:02.927684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703376d-6dc0-457e-9847-a682fa9b4fc6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:02.933335 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5500f9-c764-4280-9070-52d6960d5c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:02.934023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5500f9-c764-4280-9070-52d6960d5c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:02.934298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5500f9-c764-4280-9070-52d6960d5c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:02.934591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5500f9-c764-4280-9070-52d6960d5c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:02.935278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5500f9-c764-4280-9070-52d6960d5c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:02.937287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5500f9-c764-4280-9070-52d6960d5c66 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:02.965224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 909/1814] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:02 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:02.970832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df1d67-619a-4b36-8bdd-4b88681a12b2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:02.980922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df1d67-619a-4b36-8bdd-4b88681a12b2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.045551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4439bc-1861-427e-ac51-e5fa420818ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.046403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4439bc-1861-427e-ac51-e5fa420818ec admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.046741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4439bc-1861-427e-ac51-e5fa420818ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.047119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4439bc-1861-427e-ac51-e5fa420818ec admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.048175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4439bc-1861-427e-ac51-e5fa420818ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.050736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4439bc-1861-427e-ac51-e5fa420818ec admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.072367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 906/1815] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:02 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:03.076782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5500f9-c764-4280-9070-52d6960d5c66 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:03.087743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5500f9-c764-4280-9070-52d6960d5c66 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.093321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db550c4-3126-4110-ab62-bc9ebd669c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.093992 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db550c4-3126-4110-ab62-bc9ebd669c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.094273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db550c4-3126-4110-ab62-bc9ebd669c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.094550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db550c4-3126-4110-ab62-bc9ebd669c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.095287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db550c4-3126-4110-ab62-bc9ebd669c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.096957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db550c4-3126-4110-ab62-bc9ebd669c11 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.107644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 910/1816] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:03.111860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4439bc-1861-427e-ac51-e5fa420818ec admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:03.123201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4439bc-1861-427e-ac51-e5fa420818ec admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.133335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933dcc3-d361-4faa-b22c-dd8e1f0b1206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.134090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933dcc3-d361-4faa-b22c-dd8e1f0b1206 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.134380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933dcc3-d361-4faa-b22c-dd8e1f0b1206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.134650 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933dcc3-d361-4faa-b22c-dd8e1f0b1206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.135420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933dcc3-d361-4faa-b22c-dd8e1f0b1206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.137057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933dcc3-d361-4faa-b22c-dd8e1f0b1206 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.148342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 907/1817] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:03.152567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db550c4-3126-4110-ab62-bc9ebd669c11 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:03.164800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db550c4-3126-4110-ab62-bc9ebd669c11 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.172081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52457a40-4065-49a7-8fdb-43d6ef7430f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.173254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457a40-4065-49a7-8fdb-43d6ef7430f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.173456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457a40-4065-49a7-8fdb-43d6ef7430f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.173803 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457a40-4065-49a7-8fdb-43d6ef7430f4 admin admin] PATH_INFO: `/v3/projects/12fe81d95da84a24a4527bbdcc6fc5a2/users/316875824f6e4744ae3c8e257bae54d2/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.196436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52457a40-4065-49a7-8fdb-43d6ef7430f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12fe81d95da84a24a4527bbdcc6fc5a2, user_id=316875824f6e4744ae3c8e257bae54d2, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.199101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52457a40-4065-49a7-8fdb-43d6ef7430f4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.215637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 911/1818] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:03 2022] PUT /identity/v3/projects/12fe81d95da84a24a4527bbdcc6fc5a2/users/316875824f6e4744ae3c8e257bae54d2/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:03.228944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bd575-1d19-46ad-b86f-28e50a4d73e6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.229196 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bd575-1d19-46ad-b86f-28e50a4d73e6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.229465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bd575-1d19-46ad-b86f-28e50a4d73e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.257875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f10bd575-1d19-46ad-b86f-28e50a4d73e6 None None] MFA Rules not processed for user `316875824f6e4744ae3c8e257bae54d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:03.259127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f10bd575-1d19-46ad-b86f-28e50a4d73e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:03.331997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 908/1819] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:03.336598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52457a40-4065-49a7-8fdb-43d6ef7430f4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:03.354838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:03.400533 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52457a40-4065-49a7-8fdb-43d6ef7430f4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.405362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce518d-9fba-4f4d-ad6c-a24d54ac37f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.405742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce518d-9fba-4f4d-ad6c-a24d54ac37f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.406021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce518d-9fba-4f4d-ad6c-a24d54ac37f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.406293 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce518d-9fba-4f4d-ad6c-a24d54ac37f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.406983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce518d-9fba-4f4d-ad6c-a24d54ac37f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.408923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce518d-9fba-4f4d-ad6c-a24d54ac37f2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.435821 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 912/1820] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:03 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:03.439980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bd575-1d19-46ad-b86f-28e50a4d73e6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:03.451683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bd575-1d19-46ad-b86f-28e50a4d73e6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.460297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:03.515042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ab7ee-65c8-4ba6-b9a6-483b7a4e3d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.515722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab7ee-65c8-4ba6-b9a6-483b7a4e3d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.516005 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab7ee-65c8-4ba6-b9a6-483b7a4e3d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.516353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab7ee-65c8-4ba6-b9a6-483b7a4e3d0b admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.517088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ab7ee-65c8-4ba6-b9a6-483b7a4e3d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.519048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ab7ee-65c8-4ba6-b9a6-483b7a4e3d0b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.539508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 909/1821] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:03 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:03.543618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce518d-9fba-4f4d-ad6c-a24d54ac37f2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:03.553965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce518d-9fba-4f4d-ad6c-a24d54ac37f2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.558846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92252b7-8b49-4c13-b2d8-8c7f639c77c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.559508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92252b7-8b49-4c13-b2d8-8c7f639c77c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.560247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92252b7-8b49-4c13-b2d8-8c7f639c77c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.560247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92252b7-8b49-4c13-b2d8-8c7f639c77c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.560791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92252b7-8b49-4c13-b2d8-8c7f639c77c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.562421 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92252b7-8b49-4c13-b2d8-8c7f639c77c8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.573155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 913/1822] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:03.577425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ab7ee-65c8-4ba6-b9a6-483b7a4e3d0b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:03.587699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ab7ee-65c8-4ba6-b9a6-483b7a4e3d0b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.592087 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad592e-5525-42ef-9197-e2d3c49a3775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.593029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad592e-5525-42ef-9197-e2d3c49a3775 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.593307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad592e-5525-42ef-9197-e2d3c49a3775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.593577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad592e-5525-42ef-9197-e2d3c49a3775 admin admin] PATH_INFO: `/v3/projects/5082d51646a8405480d60194e3192b0d/users/7ef52a50d3d14d5b89eb1c1700cfbb40/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.608759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad592e-5525-42ef-9197-e2d3c49a3775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5082d51646a8405480d60194e3192b0d, user_id=7ef52a50d3d14d5b89eb1c1700cfbb40, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.611528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad592e-5525-42ef-9197-e2d3c49a3775 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.627274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 910/1823] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:03 2022] PUT /identity/v3/projects/5082d51646a8405480d60194e3192b0d/users/7ef52a50d3d14d5b89eb1c1700cfbb40/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:03.631990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92252b7-8b49-4c13-b2d8-8c7f639c77c8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:03.642119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92252b7-8b49-4c13-b2d8-8c7f639c77c8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.703061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c7008-fdfd-4311-9dac-3306b0524660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.703759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c7008-fdfd-4311-9dac-3306b0524660 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.704036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c7008-fdfd-4311-9dac-3306b0524660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.704347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c7008-fdfd-4311-9dac-3306b0524660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.705080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c7008-fdfd-4311-9dac-3306b0524660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.706893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c7008-fdfd-4311-9dac-3306b0524660 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.717529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 914/1824] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:03.721665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad592e-5525-42ef-9197-e2d3c49a3775 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:03.732609 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad592e-5525-42ef-9197-e2d3c49a3775 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.738226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadb1d7-0f6a-418f-927b-dd894091b6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.739125 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadb1d7-0f6a-418f-927b-dd894091b6c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.739407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadb1d7-0f6a-418f-927b-dd894091b6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.739679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadb1d7-0f6a-418f-927b-dd894091b6c4 admin admin] PATH_INFO: `/v3/projects/5082d51646a8405480d60194e3192b0d/users/7ef52a50d3d14d5b89eb1c1700cfbb40/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.741730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dadb1d7-0f6a-418f-927b-dd894091b6c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5082d51646a8405480d60194e3192b0d, user_id=7ef52a50d3d14d5b89eb1c1700cfbb40, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.744397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dadb1d7-0f6a-418f-927b-dd894091b6c4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.755337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 911/1825] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:03 2022] PUT /identity/v3/projects/5082d51646a8405480d60194e3192b0d/users/7ef52a50d3d14d5b89eb1c1700cfbb40/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:03.766218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b420eb-dd4c-47bc-a2e9-fab42b97b8ac None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.766396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b420eb-dd4c-47bc-a2e9-fab42b97b8ac None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.766671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b420eb-dd4c-47bc-a2e9-fab42b97b8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.797062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-70b420eb-dd4c-47bc-a2e9-fab42b97b8ac None None] MFA Rules not processed for user `7ef52a50d3d14d5b89eb1c1700cfbb40`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:03.798271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-70b420eb-dd4c-47bc-a2e9-fab42b97b8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:03.872689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 915/1826] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:03.895270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadb1d7-0f6a-418f-927b-dd894091b6c4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:03.897054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3dadb1d7-0f6a-418f-927b-dd894091b6c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:03.956348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadb1d7-0f6a-418f-927b-dd894091b6c4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:03.960138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65aef5-d5ed-46c5-a3aa-94acb62be291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:03.960790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65aef5-d5ed-46c5-a3aa-94acb62be291 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:03.961074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65aef5-d5ed-46c5-a3aa-94acb62be291 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:03.961355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65aef5-d5ed-46c5-a3aa-94acb62be291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:03.970255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65aef5-d5ed-46c5-a3aa-94acb62be291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:03.971729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65aef5-d5ed-46c5-a3aa-94acb62be291 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:03.980984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 912/1827] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:03 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:04.030556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b420eb-dd4c-47bc-a2e9-fab42b97b8ac None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:04.040755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b420eb-dd4c-47bc-a2e9-fab42b97b8ac None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:04.044705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9a9b-0aef-4b66-9354-1229a25fd8cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:04.045343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9a9b-0aef-4b66-9354-1229a25fd8cf service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:04.045626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9a9b-0aef-4b66-9354-1229a25fd8cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:04.045902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9a9b-0aef-4b66-9354-1229a25fd8cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:04.055764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:04.111142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9a9b-0aef-4b66-9354-1229a25fd8cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:04.112665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9a9b-0aef-4b66-9354-1229a25fd8cf service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:04.122463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 916/1828] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:05.646040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65aef5-d5ed-46c5-a3aa-94acb62be291 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:05.647670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ee65aef5-d5ed-46c5-a3aa-94acb62be291 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:05.721255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65aef5-d5ed-46c5-a3aa-94acb62be291 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:05.729184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:05.730161 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:05.730476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:05.730812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] PATH_INFO: `/v3/users/32b848a140014907b1db2e75232ac7b3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:05.734110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b848a140014907b1db2e75232ac7b3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:05.736198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:05.743898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:05.808960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b848a140014907b1db2e75232ac7b3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:05.809872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b848a140014907b1db2e75232ac7b3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:05.819511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b848a140014907b1db2e75232ac7b3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:05.820289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b848a140014907b1db2e75232ac7b3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:05.848645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b848a140014907b1db2e75232ac7b3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:05.873123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 913/1829] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:05 2022] DELETE /identity/v3/users/32b848a140014907b1db2e75232ac7b3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:05.875734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fd5f0-7689-4308-b3e6-d7d422089a61 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:05.876317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fd5f0-7689-4308-b3e6-d7d422089a61 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:05.876317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fd5f0-7689-4308-b3e6-d7d422089a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:05.900436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0c8fd5f0-7689-4308-b3e6-d7d422089a61 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:05.901952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c8fd5f0-7689-4308-b3e6-d7d422089a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:05.978247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 917/1830] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:05.988873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:05.990460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:06.050376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487a954-64a6-4670-8b67-07ba960596e5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:06.054019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192955c-ed17-4512-98e1-ea7f2e878fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:06.054694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192955c-ed17-4512-98e1-ea7f2e878fce service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:06.054978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192955c-ed17-4512-98e1-ea7f2e878fce service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:06.055276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192955c-ed17-4512-98e1-ea7f2e878fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:06.064958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192955c-ed17-4512-98e1-ea7f2e878fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:06.066454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192955c-ed17-4512-98e1-ea7f2e878fce service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:06.076409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 914/1831] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:06.387847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fd5f0-7689-4308-b3e6-d7d422089a61 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:06.389547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c8fd5f0-7689-4308-b3e6-d7d422089a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:06.402614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fd5f0-7689-4308-b3e6-d7d422089a61 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:06.408422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:06.409440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:06.409716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:06.409992 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] PATH_INFO: `/v3/projects/3c887532702d4c20baddf1f710c5a762` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:06.411306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c887532702d4c20baddf1f710c5a762)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:06.413355 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:06.471061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c887532702d4c20baddf1f710c5a762', 'request_id': 'req-b43266a4-d878-4b36-965e-c34ed7cc9691'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:06.477709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c887532702d4c20baddf1f710c5a762', 'request_id': 'req-b43266a4-d878-4b36-965e-c34ed7cc9691'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:06.488038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c887532702d4c20baddf1f710c5a762', 'request_id': 'req-b43266a4-d878-4b36-965e-c34ed7cc9691'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:06.499756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] The token cache is being invalidate because project 3c887532702d4c20baddf1f710c5a762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:06.500047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:06.500740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:06.502323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 918/1832] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:06 2022] DELETE /identity/v3/projects/3c887532702d4c20baddf1f710c5a762 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:06.506730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192955c-ed17-4512-98e1-ea7f2e878fce service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:06.518635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192955c-ed17-4512-98e1-ea7f2e878fce service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:06.575441 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:06.576360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:06.576644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:06.576903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] PATH_INFO: `/v3/users/2e2fb8d844bb4974aa4a8956763a91dc` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:06.578258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2fb8d844bb4974aa4a8956763a91dc)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:06.580298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:06.616796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:06.631299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2fb8d844bb4974aa4a8956763a91dc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:06.631948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2fb8d844bb4974aa4a8956763a91dc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:06.639465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2fb8d844bb4974aa4a8956763a91dc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:06.640176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2fb8d844bb4974aa4a8956763a91dc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:06.652734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-13e6e872-feea-4403-bd02-4a4b553e6e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2fb8d844bb4974aa4a8956763a91dc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:06.667764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 915/1833] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:06 2022] DELETE /identity/v3/users/2e2fb8d844bb4974aa4a8956763a91dc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:07.035729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:07.037283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:07.051683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43266a4-d878-4b36-965e-c34ed7cc9691 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:07.089134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:07.128365 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:07.129319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.129600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.129870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] PATH_INFO: `/v3/projects/05a174544d44430d98de09045c03336e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.131763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a174544d44430d98de09045c03336e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:07.134004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:07.218182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a174544d44430d98de09045c03336e', 'request_id': 'req-404b81b7-5914-41c1-9cb7-8a23dc5cd975'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:07.226049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a174544d44430d98de09045c03336e', 'request_id': 'req-404b81b7-5914-41c1-9cb7-8a23dc5cd975'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:07.245947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a174544d44430d98de09045c03336e', 'request_id': 'req-404b81b7-5914-41c1-9cb7-8a23dc5cd975'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:07.257922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] The token cache is being invalidate because project 05a174544d44430d98de09045c03336e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:07.258292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:07.259016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-404b81b7-5914-41c1-9cb7-8a23dc5cd975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:07.260917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 919/1834] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:07 2022] DELETE /identity/v3/projects/05a174544d44430d98de09045c03336e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:07.266011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80612618-cfd4-4975-888d-1ed3eba64a22 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.267271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80612618-cfd4-4975-888d-1ed3eba64a22 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.267735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80612618-cfd4-4975-888d-1ed3eba64a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.291300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-80612618-cfd4-4975-888d-1ed3eba64a22 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:07.292916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-80612618-cfd4-4975-888d-1ed3eba64a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:07.403368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 916/1835] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:07.411197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a46d2-3c9e-415a-8124-63cdd94ca42e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.411478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a46d2-3c9e-415a-8124-63cdd94ca42e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.411990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a46d2-3c9e-415a-8124-63cdd94ca42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.436411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8b8a46d2-3c9e-415a-8124-63cdd94ca42e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:07.438527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8b8a46d2-3c9e-415a-8124-63cdd94ca42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:07.452558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 920/1836] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:07.454553 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80612618-cfd4-4975-888d-1ed3eba64a22 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:07.472037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80612618-cfd4-4975-888d-1ed3eba64a22 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:07.481167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e912b-f9a2-454d-8c8d-7669c7bffd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:07.482402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e912b-f9a2-454d-8c8d-7669c7bffd56 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.483085 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e912b-f9a2-454d-8c8d-7669c7bffd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.483599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e912b-f9a2-454d-8c8d-7669c7bffd56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.484599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e912b-f9a2-454d-8c8d-7669c7bffd56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:07.485108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e912b-f9a2-454d-8c8d-7669c7bffd56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:07.489385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e912b-f9a2-454d-8c8d-7669c7bffd56 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:07.517046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 917/1837] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:07.518011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a46d2-3c9e-415a-8124-63cdd94ca42e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:07.531010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a46d2-3c9e-415a-8124-63cdd94ca42e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:07.536972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb0c25-bf15-407e-ba61-925edb139759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:07.538798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb0c25-bf15-407e-ba61-925edb139759 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.539585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb0c25-bf15-407e-ba61-925edb139759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.540034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb0c25-bf15-407e-ba61-925edb139759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.541681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb0c25-bf15-407e-ba61-925edb139759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:07.544796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb0c25-bf15-407e-ba61-925edb139759 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:07.577807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 921/1838] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:07 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:07.579027 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e912b-f9a2-454d-8c8d-7669c7bffd56 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:07.590623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e912b-f9a2-454d-8c8d-7669c7bffd56 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:07.665138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ad0e7-eec3-47cf-b96e-8d158e73884b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:07.666079 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ad0e7-eec3-47cf-b96e-8d158e73884b admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.666474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ad0e7-eec3-47cf-b96e-8d158e73884b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.666811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ad0e7-eec3-47cf-b96e-8d158e73884b admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.667615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ad0e7-eec3-47cf-b96e-8d158e73884b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:07.669670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ad0e7-eec3-47cf-b96e-8d158e73884b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:07.691408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 918/1839] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:07 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:07.695503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb0c25-bf15-407e-ba61-925edb139759 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:07.706199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb0c25-bf15-407e-ba61-925edb139759 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:07.711125 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fc7b6-c418-4ee2-aa90-28b2c7b8a6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:07.711925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fc7b6-c418-4ee2-aa90-28b2c7b8a6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.712204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fc7b6-c418-4ee2-aa90-28b2c7b8a6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.712474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fc7b6-c418-4ee2-aa90-28b2c7b8a6d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.713197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fc7b6-c418-4ee2-aa90-28b2c7b8a6d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:07.714811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fc7b6-c418-4ee2-aa90-28b2c7b8a6d4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:07.726226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 922/1840] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:07.730392 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ad0e7-eec3-47cf-b96e-8d158e73884b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:07.741030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ad0e7-eec3-47cf-b96e-8d158e73884b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:07.747420 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d164679-0c47-45af-b32a-7531931ac2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:07.748581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d164679-0c47-45af-b32a-7531931ac2ed admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.748872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d164679-0c47-45af-b32a-7531931ac2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.749149 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d164679-0c47-45af-b32a-7531931ac2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.749902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d164679-0c47-45af-b32a-7531931ac2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:07.751673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d164679-0c47-45af-b32a-7531931ac2ed admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:07.763490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 919/1841] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:07.767957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fc7b6-c418-4ee2-aa90-28b2c7b8a6d4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:07.779317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fc7b6-c418-4ee2-aa90-28b2c7b8a6d4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:07.785646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d61ed-cf26-4393-874a-688ad6963f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:07.786737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d61ed-cf26-4393-874a-688ad6963f78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.787018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d61ed-cf26-4393-874a-688ad6963f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.787864 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d61ed-cf26-4393-874a-688ad6963f78 admin admin] PATH_INFO: `/v3/projects/3bac9fcb27124cd98859c630d1bc6b4f/users/b1dcf19121bd44b1907d0af581b8c6bd/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.803482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d61ed-cf26-4393-874a-688ad6963f78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bac9fcb27124cd98859c630d1bc6b4f, user_id=b1dcf19121bd44b1907d0af581b8c6bd, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:07.806205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d61ed-cf26-4393-874a-688ad6963f78 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:07.821319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 923/1842] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:07 2022] PUT /identity/v3/projects/3bac9fcb27124cd98859c630d1bc6b4f/users/b1dcf19121bd44b1907d0af581b8c6bd/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:07.833335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d974d-6ef1-4f25-8193-20a6b1fb0444 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.833637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d974d-6ef1-4f25-8193-20a6b1fb0444 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.833953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d974d-6ef1-4f25-8193-20a6b1fb0444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.859994 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:07.863952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6b9d974d-6ef1-4f25-8193-20a6b1fb0444 None None] MFA Rules not processed for user `b1dcf19121bd44b1907d0af581b8c6bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:07.868579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6b9d974d-6ef1-4f25-8193-20a6b1fb0444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:07.890025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d61ed-cf26-4393-874a-688ad6963f78 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:07.891836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-923d61ed-cf26-4393-874a-688ad6963f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:07.946585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 920/1843] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:07.947813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d974d-6ef1-4f25-8193-20a6b1fb0444 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:07.962186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d61ed-cf26-4393-874a-688ad6963f78 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:07.967185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:07.968280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:07.968569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:07.969060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] PATH_INFO: `/v3/users/316875824f6e4744ae3c8e257bae54d2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:07.970711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=316875824f6e4744ae3c8e257bae54d2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:07.973158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:07.975314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:08.023667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d974d-6ef1-4f25-8193-20a6b1fb0444 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:08.029265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251a9c8-209c-4d51-98e4-4a64948b473a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:08.030022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251a9c8-209c-4d51-98e4-4a64948b473a admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.030231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251a9c8-209c-4d51-98e4-4a64948b473a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.030504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251a9c8-209c-4d51-98e4-4a64948b473a admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.031255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251a9c8-209c-4d51-98e4-4a64948b473a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:08.033175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251a9c8-209c-4d51-98e4-4a64948b473a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:08.041159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '316875824f6e4744ae3c8e257bae54d2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:08.049117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '316875824f6e4744ae3c8e257bae54d2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:08.049917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '316875824f6e4744ae3c8e257bae54d2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:08.058792 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '316875824f6e4744ae3c8e257bae54d2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:08.066182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 921/1844] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:07 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:08.067273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251a9c8-209c-4d51-98e4-4a64948b473a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:08.071499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-791c5882-07a1-4471-9d63-105f23b37f7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '316875824f6e4744ae3c8e257bae54d2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:08.079266 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 924/1845] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:07 2022] DELETE /identity/v3/users/316875824f6e4744ae3c8e257bae54d2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:08.084292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224b4f7-b950-49ec-8cbc-58de9c7ecb35 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.084788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224b4f7-b950-49ec-8cbc-58de9c7ecb35 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.085259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224b4f7-b950-49ec-8cbc-58de9c7ecb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.104723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8224b4f7-b950-49ec-8cbc-58de9c7ecb35 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:08.105949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8224b4f7-b950-49ec-8cbc-58de9c7ecb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:08.134272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251a9c8-209c-4d51-98e4-4a64948b473a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:08.166074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:08.168379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 925/1846] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:08.179208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224b4f7-b950-49ec-8cbc-58de9c7ecb35 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:08.180922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8224b4f7-b950-49ec-8cbc-58de9c7ecb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:08.194115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bef898-4dde-47e7-9fb7-ba81036988d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:08.194808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bef898-4dde-47e7-9fb7-ba81036988d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.195091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bef898-4dde-47e7-9fb7-ba81036988d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.195369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bef898-4dde-47e7-9fb7-ba81036988d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.196095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bef898-4dde-47e7-9fb7-ba81036988d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:08.198058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bef898-4dde-47e7-9fb7-ba81036988d1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:08.229877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 922/1847] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:08 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:08.234438 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bef898-4dde-47e7-9fb7-ba81036988d1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:08.236342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-05bef898-4dde-47e7-9fb7-ba81036988d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:08.248809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bef898-4dde-47e7-9fb7-ba81036988d1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:08.248990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224b4f7-b950-49ec-8cbc-58de9c7ecb35 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:08.252562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a13914-4c2f-4b3f-8022-ce3e9494c056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:08.253291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a13914-4c2f-4b3f-8022-ce3e9494c056 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.253575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a13914-4c2f-4b3f-8022-ce3e9494c056 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.253859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a13914-4c2f-4b3f-8022-ce3e9494c056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.256528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643ec45-e8e2-4833-b46c-b70364390ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:08.257362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643ec45-e8e2-4833-b46c-b70364390ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.258366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643ec45-e8e2-4833-b46c-b70364390ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.258947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643ec45-e8e2-4833-b46c-b70364390ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.260456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643ec45-e8e2-4833-b46c-b70364390ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:08.262231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643ec45-e8e2-4833-b46c-b70364390ec1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:08.263709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a13914-4c2f-4b3f-8022-ce3e9494c056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:08.265206 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a13914-4c2f-4b3f-8022-ce3e9494c056 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:08.274509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 923/1848] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:08.277365 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 926/1849] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:08.278720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643ec45-e8e2-4833-b46c-b70364390ec1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:08.289108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643ec45-e8e2-4833-b46c-b70364390ec1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:08.293755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93a17f-0156-48a3-afb4-3538aef547e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:08.294683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93a17f-0156-48a3-afb4-3538aef547e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.294995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93a17f-0156-48a3-afb4-3538aef547e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.295258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93a17f-0156-48a3-afb4-3538aef547e1 admin admin] PATH_INFO: `/v3/projects/aadd9ed106c246399fb408cf7c73ef04/users/02b27faf363d44b38ed844cee2344148/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.310691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93a17f-0156-48a3-afb4-3538aef547e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aadd9ed106c246399fb408cf7c73ef04, user_id=02b27faf363d44b38ed844cee2344148, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:08.313401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93a17f-0156-48a3-afb4-3538aef547e1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:08.327729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 924/1850] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:08 2022] PUT /identity/v3/projects/aadd9ed106c246399fb408cf7c73ef04/users/02b27faf363d44b38ed844cee2344148/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:08.331958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a13914-4c2f-4b3f-8022-ce3e9494c056 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:08.342351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a13914-4c2f-4b3f-8022-ce3e9494c056 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:08.364706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:08.432064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f3700-b942-479c-90d3-921d73c41b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:08.433195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f3700-b942-479c-90d3-921d73c41b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.434140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f3700-b942-479c-90d3-921d73c41b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.434427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f3700-b942-479c-90d3-921d73c41b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.435275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f3700-b942-479c-90d3-921d73c41b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:08.436966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f3700-b942-479c-90d3-921d73c41b8f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:08.447857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 927/1851] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:08.452064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93a17f-0156-48a3-afb4-3538aef547e1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:08.465491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93a17f-0156-48a3-afb4-3538aef547e1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:08.469946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35028ce7-4325-46d4-997a-4bf2442eae01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:08.470431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35028ce7-4325-46d4-997a-4bf2442eae01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.470719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35028ce7-4325-46d4-997a-4bf2442eae01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.470993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35028ce7-4325-46d4-997a-4bf2442eae01 admin admin] PATH_INFO: `/v3/projects/aadd9ed106c246399fb408cf7c73ef04/users/02b27faf363d44b38ed844cee2344148/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.473344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35028ce7-4325-46d4-997a-4bf2442eae01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aadd9ed106c246399fb408cf7c73ef04, user_id=02b27faf363d44b38ed844cee2344148, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:08.476206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35028ce7-4325-46d4-997a-4bf2442eae01 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:08.493870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 925/1852] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:08 2022] PUT /identity/v3/projects/aadd9ed106c246399fb408cf7c73ef04/users/02b27faf363d44b38ed844cee2344148/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:08.505500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ec552-259c-48c0-a1b8-cb092f5ed309 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.505789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ec552-259c-48c0-a1b8-cb092f5ed309 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.506060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ec552-259c-48c0-a1b8-cb092f5ed309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.541345 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8c8ec552-259c-48c0-a1b8-cb092f5ed309 None None] MFA Rules not processed for user `02b27faf363d44b38ed844cee2344148`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:08.542736 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c8ec552-259c-48c0-a1b8-cb092f5ed309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:08.629349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35028ce7-4325-46d4-997a-4bf2442eae01 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:08.631079 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-35028ce7-4325-46d4-997a-4bf2442eae01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:08.718045 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 928/1853] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:08.734465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35028ce7-4325-46d4-997a-4bf2442eae01 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:08.739507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:08.740419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.740700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.740972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] PATH_INFO: `/v3/projects/12fe81d95da84a24a4527bbdcc6fc5a2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.742157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12fe81d95da84a24a4527bbdcc6fc5a2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:08.744128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:08.820264 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12fe81d95da84a24a4527bbdcc6fc5a2', 'request_id': 'req-d197d67b-b003-4bd8-b677-7f1057081514'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:08.837721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12fe81d95da84a24a4527bbdcc6fc5a2', 'request_id': 'req-d197d67b-b003-4bd8-b677-7f1057081514'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:08.849097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12fe81d95da84a24a4527bbdcc6fc5a2', 'request_id': 'req-d197d67b-b003-4bd8-b677-7f1057081514'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:08.867043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ec552-259c-48c0-a1b8-cb092f5ed309 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:08.868210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c8ec552-259c-48c0-a1b8-cb092f5ed309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:08.873825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] The token cache is being invalidate because project 12fe81d95da84a24a4527bbdcc6fc5a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:08.874102 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:08.874824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:08.876684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 926/1854] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:08 2022] DELETE /identity/v3/projects/12fe81d95da84a24a4527bbdcc6fc5a2 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:08.881036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:08.882758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:08.896048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:08.933511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ec552-259c-48c0-a1b8-cb092f5ed309 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:08.973162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197d67b-b003-4bd8-b677-7f1057081514 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:08.981943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:08.982873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.983214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.983500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] PATH_INFO: `/v3/users/7ef52a50d3d14d5b89eb1c1700cfbb40` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.984849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ef52a50d3d14d5b89eb1c1700cfbb40)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:08.986813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:08.994406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6f7cb-8e8f-4a03-9b1a-ae9cec34151c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:08.995108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6f7cb-8e8f-4a03-9b1a-ae9cec34151c service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:08.995404 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6f7cb-8e8f-4a03-9b1a-ae9cec34151c service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:08.995706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6f7cb-8e8f-4a03-9b1a-ae9cec34151c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:08.998624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2df6f7cb-8e8f-4a03-9b1a-ae9cec34151c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:09.013702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:09.048749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ef52a50d3d14d5b89eb1c1700cfbb40'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.049343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ef52a50d3d14d5b89eb1c1700cfbb40'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.059160 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ef52a50d3d14d5b89eb1c1700cfbb40'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.059851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ef52a50d3d14d5b89eb1c1700cfbb40'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.069608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6f7cb-8e8f-4a03-9b1a-ae9cec34151c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:09.071280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6f7cb-8e8f-4a03-9b1a-ae9cec34151c service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:09.075082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ef52a50d3d14d5b89eb1c1700cfbb40'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.081958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 929/1855] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:09.091736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 927/1856] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:08 2022] DELETE /identity/v3/users/7ef52a50d3d14d5b89eb1c1700cfbb40 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:09.496799 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6f7cb-8e8f-4a03-9b1a-ae9cec34151c service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:09.510393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6f7cb-8e8f-4a03-9b1a-ae9cec34151c service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:09.599065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:09.599674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:09.600086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:09.600393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] PATH_INFO: `/v3/projects/5082d51646a8405480d60194e3192b0d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:09.601785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5082d51646a8405480d60194e3192b0d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:09.603729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:09.647819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:09.650529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:09.665559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c6a18-b853-40a7-af0d-f4491fe5a929 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:09.669863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5082d51646a8405480d60194e3192b0d', 'request_id': 'req-b95c050f-4b89-40de-b94a-36681ab8aa3e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.677949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5082d51646a8405480d60194e3192b0d', 'request_id': 'req-b95c050f-4b89-40de-b94a-36681ab8aa3e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.694087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5082d51646a8405480d60194e3192b0d', 'request_id': 'req-b95c050f-4b89-40de-b94a-36681ab8aa3e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.708011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] The token cache is being invalidate because project 5082d51646a8405480d60194e3192b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:09.708615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.709264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b95c050f-4b89-40de-b94a-36681ab8aa3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.715436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 930/1857] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:09 2022] DELETE /identity/v3/projects/5082d51646a8405480d60194e3192b0d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:09.717739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395be85b-57e7-47eb-9e36-7c095c50c7a6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:09.718108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395be85b-57e7-47eb-9e36-7c095c50c7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:09.718387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395be85b-57e7-47eb-9e36-7c095c50c7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:09.741888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-395be85b-57e7-47eb-9e36-7c095c50c7a6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:09.743205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-395be85b-57e7-47eb-9e36-7c095c50c7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:09.748862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:09.750011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:09.750331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:09.750600 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] PATH_INFO: `/v3/users/b1dcf19121bd44b1907d0af581b8c6bd` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:09.752198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1dcf19121bd44b1907d0af581b8c6bd)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:09.752801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 931/1858] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:09.755192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:09.766899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3d5d1-6c48-4c2c-9ee9-d947435283dd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:09.767186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3d5d1-6c48-4c2c-9ee9-d947435283dd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:09.767460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3d5d1-6c48-4c2c-9ee9-d947435283dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:09.786115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2eb3d5d1-6c48-4c2c-9ee9-d947435283dd None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:09.787313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2eb3d5d1-6c48-4c2c-9ee9-d947435283dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:09.787605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:09.795841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 932/1859] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:09.800890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3d5d1-6c48-4c2c-9ee9-d947435283dd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:09.808031 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1dcf19121bd44b1907d0af581b8c6bd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.808693 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1dcf19121bd44b1907d0af581b8c6bd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.811196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3d5d1-6c48-4c2c-9ee9-d947435283dd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:09.815880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928fd2d-dab9-4fc7-8661-ca3c2c681261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:09.816586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928fd2d-dab9-4fc7-8661-ca3c2c681261 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:09.816869 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928fd2d-dab9-4fc7-8661-ca3c2c681261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:09.817172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928fd2d-dab9-4fc7-8661-ca3c2c681261 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:09.817936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928fd2d-dab9-4fc7-8661-ca3c2c681261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:09.818248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928fd2d-dab9-4fc7-8661-ca3c2c681261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:09.819871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928fd2d-dab9-4fc7-8661-ca3c2c681261 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:09.823703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1dcf19121bd44b1907d0af581b8c6bd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.824487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1dcf19121bd44b1907d0af581b8c6bd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.833832 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 933/1860] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:58:09.838620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928fd2d-dab9-4fc7-8661-ca3c2c681261 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:09.840648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c928fd2d-dab9-4fc7-8661-ca3c2c681261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:09.849884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-728248dc-9f4e-4ab7-923d-5dcb9a67c154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1dcf19121bd44b1907d0af581b8c6bd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:09.854157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928fd2d-dab9-4fc7-8661-ca3c2c681261 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:09.860134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad665-4605-40de-9e4d-9fafc2dbebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:09.861167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad665-4605-40de-9e4d-9fafc2dbebab admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:09.861740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad665-4605-40de-9e4d-9fafc2dbebab admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:09.862078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad665-4605-40de-9e4d-9fafc2dbebab admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:09.863004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aad665-4605-40de-9e4d-9fafc2dbebab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:09.865469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aad665-4605-40de-9e4d-9fafc2dbebab admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:09.866659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 928/1861] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:09 2022] DELETE /identity/v3/users/b1dcf19121bd44b1907d0af581b8c6bd => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:09.871455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41905b1-a0d0-406b-b73c-1f40b2538778 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:09.871751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41905b1-a0d0-406b-b73c-1f40b2538778 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:09.872204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41905b1-a0d0-406b-b73c-1f40b2538778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:09.877501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:09.890857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d41905b1-a0d0-406b-b73c-1f40b2538778 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:09.892169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d41905b1-a0d0-406b-b73c-1f40b2538778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:09.900168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 934/1862] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:09 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:09.904082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad665-4605-40de-9e4d-9fafc2dbebab admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:09.957958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 929/1863] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:09.968537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41905b1-a0d0-406b-b73c-1f40b2538778 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:09.969933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d41905b1-a0d0-406b-b73c-1f40b2538778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:09.973917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad665-4605-40de-9e4d-9fafc2dbebab admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:09.978697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed8764-b4a1-4c48-abaa-81bb3c2839b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:09.979391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed8764-b4a1-4c48-abaa-81bb3c2839b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:09.980168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed8764-b4a1-4c48-abaa-81bb3c2839b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:09.980168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed8764-b4a1-4c48-abaa-81bb3c2839b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:09.980686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed8764-b4a1-4c48-abaa-81bb3c2839b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:09.982641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed8764-b4a1-4c48-abaa-81bb3c2839b1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.006015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 935/1864] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:09 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:10.010135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed8764-b4a1-4c48-abaa-81bb3c2839b1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.020099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed8764-b4a1-4c48-abaa-81bb3c2839b1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.024618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd142f1-f7fa-498b-987a-83607bb1fe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.025286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd142f1-f7fa-498b-987a-83607bb1fe2e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.025659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd142f1-f7fa-498b-987a-83607bb1fe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.025939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd142f1-f7fa-498b-987a-83607bb1fe2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.026667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd142f1-f7fa-498b-987a-83607bb1fe2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.028284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd142f1-f7fa-498b-987a-83607bb1fe2e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.034601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41905b1-a0d0-406b-b73c-1f40b2538778 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.040447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 936/1865] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:10.044335 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd142f1-f7fa-498b-987a-83607bb1fe2e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.054153 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd142f1-f7fa-498b-987a-83607bb1fe2e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.059575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b80a6-1950-47b2-8b9c-bf4be058ba09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.060200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b80a6-1950-47b2-8b9c-bf4be058ba09 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.060484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b80a6-1950-47b2-8b9c-bf4be058ba09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.060759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b80a6-1950-47b2-8b9c-bf4be058ba09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.061482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b80a6-1950-47b2-8b9c-bf4be058ba09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.063101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b80a6-1950-47b2-8b9c-bf4be058ba09 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.075930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 937/1866] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:10.079820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b80a6-1950-47b2-8b9c-bf4be058ba09 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.086675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f99bf-2ed9-46a7-8a79-e81fda71576b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.087471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f99bf-2ed9-46a7-8a79-e81fda71576b service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.087600 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f99bf-2ed9-46a7-8a79-e81fda71576b service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.087882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f99bf-2ed9-46a7-8a79-e81fda71576b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.089723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b80a6-1950-47b2-8b9c-bf4be058ba09 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.094275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dc88d-7728-4b7f-8f6e-261ff2ab74fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.095086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dc88d-7728-4b7f-8f6e-261ff2ab74fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.095361 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dc88d-7728-4b7f-8f6e-261ff2ab74fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.095629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dc88d-7728-4b7f-8f6e-261ff2ab74fb admin admin] PATH_INFO: `/v3/projects/bb75a0a70790432ca2cab13c875e2876/users/bf5fbc9446024ff78b4ff8fa0c6bd222/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.097717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f99bf-2ed9-46a7-8a79-e81fda71576b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.099325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f99bf-2ed9-46a7-8a79-e81fda71576b service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.109433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 930/1867] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:10.110136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dc88d-7728-4b7f-8f6e-261ff2ab74fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb75a0a70790432ca2cab13c875e2876, user_id=bf5fbc9446024ff78b4ff8fa0c6bd222, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.113153 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dc88d-7728-4b7f-8f6e-261ff2ab74fb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.131765 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 938/1868] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:10 2022] PUT /identity/v3/projects/bb75a0a70790432ca2cab13c875e2876/users/bf5fbc9446024ff78b4ff8fa0c6bd222/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:10.142002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc180da-f6bf-4b2f-93bd-9c0defbe1e8c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.142303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc180da-f6bf-4b2f-93bd-9c0defbe1e8c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.142562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc180da-f6bf-4b2f-93bd-9c0defbe1e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.149493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:10.172031 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9bc180da-f6bf-4b2f-93bd-9c0defbe1e8c None None] MFA Rules not processed for user `bf5fbc9446024ff78b4ff8fa0c6bd222`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:10.173326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9bc180da-f6bf-4b2f-93bd-9c0defbe1e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:10.263845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 931/1869] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:10.265271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dc88d-7728-4b7f-8f6e-261ff2ab74fb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.345414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dc88d-7728-4b7f-8f6e-261ff2ab74fb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.350899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59de5c-372d-4cd3-86ec-f1d76662ac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.351775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59de5c-372d-4cd3-86ec-f1d76662ac7a admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.352015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59de5c-372d-4cd3-86ec-f1d76662ac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.352389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59de5c-372d-4cd3-86ec-f1d76662ac7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.353170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59de5c-372d-4cd3-86ec-f1d76662ac7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.355224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59de5c-372d-4cd3-86ec-f1d76662ac7a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.404372 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 939/1870] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:10 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:10.408367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc180da-f6bf-4b2f-93bd-9c0defbe1e8c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.425113 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc180da-f6bf-4b2f-93bd-9c0defbe1e8c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.458726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59de5c-372d-4cd3-86ec-f1d76662ac7a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.461860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2b59de5c-372d-4cd3-86ec-f1d76662ac7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:10.518312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:10.561820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1d773-a366-4d35-a49c-f03a880128ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.562681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1d773-a366-4d35-a49c-f03a880128ac admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.562968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1d773-a366-4d35-a49c-f03a880128ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.563252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1d773-a366-4d35-a49c-f03a880128ac admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.563956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1d773-a366-4d35-a49c-f03a880128ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.566384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1d773-a366-4d35-a49c-f03a880128ac admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.593784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 932/1871] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:10 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:10.598162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1d773-a366-4d35-a49c-f03a880128ac admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.613523 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1d773-a366-4d35-a49c-f03a880128ac admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.627389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd36cd-15c9-4d91-99c9-f333472ef094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.628540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd36cd-15c9-4d91-99c9-f333472ef094 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.628969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd36cd-15c9-4d91-99c9-f333472ef094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.629332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd36cd-15c9-4d91-99c9-f333472ef094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.630086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd36cd-15c9-4d91-99c9-f333472ef094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.630546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59de5c-372d-4cd3-86ec-f1d76662ac7a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.632778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd36cd-15c9-4d91-99c9-f333472ef094 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.638976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.639875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.640195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.640474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] PATH_INFO: `/v3/projects/3bac9fcb27124cd98859c630d1bc6b4f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.641718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bac9fcb27124cd98859c630d1bc6b4f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.643797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.648192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 933/1872] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:10.652273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd36cd-15c9-4d91-99c9-f333472ef094 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.662888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd36cd-15c9-4d91-99c9-f333472ef094 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.668068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5a3a1-70bc-4f40-a8a4-0b215182cf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.669066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5a3a1-70bc-4f40-a8a4-0b215182cf54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.669411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5a3a1-70bc-4f40-a8a4-0b215182cf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.669754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5a3a1-70bc-4f40-a8a4-0b215182cf54 admin admin] PATH_INFO: `/v3/projects/8d5dc1a2520b494a8e2b400faae58e28/users/ec584808586544b89c9cd3493d76dede/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.685954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5a3a1-70bc-4f40-a8a4-0b215182cf54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5dc1a2520b494a8e2b400faae58e28, user_id=ec584808586544b89c9cd3493d76dede, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.688778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5a3a1-70bc-4f40-a8a4-0b215182cf54 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.704463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 934/1873] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:10 2022] PUT /identity/v3/projects/8d5dc1a2520b494a8e2b400faae58e28/users/ec584808586544b89c9cd3493d76dede/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:10.705631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5a3a1-70bc-4f40-a8a4-0b215182cf54 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.707457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-55e5a3a1-70bc-4f40-a8a4-0b215182cf54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:10.720280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bac9fcb27124cd98859c630d1bc6b4f', 'request_id': 'req-86c79e72-0579-446b-9cde-a39e960afcea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.727488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bac9fcb27124cd98859c630d1bc6b4f', 'request_id': 'req-86c79e72-0579-446b-9cde-a39e960afcea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.738305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bac9fcb27124cd98859c630d1bc6b4f', 'request_id': 'req-86c79e72-0579-446b-9cde-a39e960afcea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.754388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] The token cache is being invalidate because project 3bac9fcb27124cd98859c630d1bc6b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:10.754919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.755617 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.757454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 940/1874] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:10 2022] DELETE /identity/v3/projects/3bac9fcb27124cd98859c630d1bc6b4f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:10.758166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.760081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:10.765932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:10.782744 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5a3a1-70bc-4f40-a8a4-0b215182cf54 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.844174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c79e72-0579-446b-9cde-a39e960afcea admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.859641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8e831-955a-4f7a-a3eb-2293ea298a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.860386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8e831-955a-4f7a-a3eb-2293ea298a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.860676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8e831-955a-4f7a-a3eb-2293ea298a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.860942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8e831-955a-4f7a-a3eb-2293ea298a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.861691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8e831-955a-4f7a-a3eb-2293ea298a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.863371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8e831-955a-4f7a-a3eb-2293ea298a36 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.876863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 941/1875] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:10.877780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8e831-955a-4f7a-a3eb-2293ea298a36 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:10.880205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b3b8e831-955a-4f7a-a3eb-2293ea298a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:10.892498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.893917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.894397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.894570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8e831-955a-4f7a-a3eb-2293ea298a36 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:10.895032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] PATH_INFO: `/v3/users/fb10576d5c91446183e2e4afcc00218b` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.898711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb10576d5c91446183e2e4afcc00218b)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.901052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.901164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:10.902000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:10.902282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:10.902558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] PATH_INFO: `/v3/users/02b27faf363d44b38ed844cee2344148` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:10.903778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b27faf363d44b38ed844cee2344148)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:10.906566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:10.967863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:10.971532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb10576d5c91446183e2e4afcc00218b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.972465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb10576d5c91446183e2e4afcc00218b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.978759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b27faf363d44b38ed844cee2344148'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.988390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b27faf363d44b38ed844cee2344148'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.989709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b27faf363d44b38ed844cee2344148'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.989855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb10576d5c91446183e2e4afcc00218b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:10.991015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb10576d5c91446183e2e4afcc00218b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:11.004894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b27faf363d44b38ed844cee2344148'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:11.011747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-870ed1c3-7a90-4fc9-9dec-e628276117fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb10576d5c91446183e2e4afcc00218b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:11.024047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b27faf363d44b38ed844cee2344148'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:11.037491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 942/1876] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:10 2022] DELETE /identity/v3/users/02b27faf363d44b38ed844cee2344148 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:11.040061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:11.043431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:11.046394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 935/1877] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:10 2022] DELETE /identity/v3/users/fb10576d5c91446183e2e4afcc00218b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:11.048846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3055f-9a1e-4111-a966-82e77fd69a74 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:11.049053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3055f-9a1e-4111-a966-82e77fd69a74 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:11.049423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3055f-9a1e-4111-a966-82e77fd69a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:11.095130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2bb3055f-9a1e-4111-a966-82e77fd69a74 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:11.097896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2bb3055f-9a1e-4111-a966-82e77fd69a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:11.158868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7b5c2-526b-4e98-9a17-aea1734d4c63 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:11.227424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55244419-3770-4e6f-9db7-22463814fb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:11.228530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55244419-3770-4e6f-9db7-22463814fb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:11.229034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55244419-3770-4e6f-9db7-22463814fb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:11.229278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55244419-3770-4e6f-9db7-22463814fb38 admin admin] PATH_INFO: `/v3/projects/8d5dc1a2520b494a8e2b400faae58e28/users/ec584808586544b89c9cd3493d76dede/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:11.234749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55244419-3770-4e6f-9db7-22463814fb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5dc1a2520b494a8e2b400faae58e28, user_id=ec584808586544b89c9cd3493d76dede, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:11.235896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 936/1878] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:11.236730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55244419-3770-4e6f-9db7-22463814fb38 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:11.250258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 943/1879] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:11 2022] PUT /identity/v3/projects/8d5dc1a2520b494a8e2b400faae58e28/users/ec584808586544b89c9cd3493d76dede/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:11.263145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb8276-291f-4d59-8a4d-a1998f16947c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:11.263517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb8276-291f-4d59-8a4d-a1998f16947c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:11.263967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb8276-291f-4d59-8a4d-a1998f16947c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:11.273779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55244419-3770-4e6f-9db7-22463814fb38 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:11.280142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-55244419-3770-4e6f-9db7-22463814fb38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:11.323986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-87fb8276-291f-4d59-8a4d-a1998f16947c None None] MFA Rules not processed for user `ec584808586544b89c9cd3493d76dede`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:11.325364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-87fb8276-291f-4d59-8a4d-a1998f16947c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:11.330459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:11.416774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55244419-3770-4e6f-9db7-22463814fb38 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:11.421952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848137f7-ed2f-4530-8c7b-25e0fa780c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:11.422620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848137f7-ed2f-4530-8c7b-25e0fa780c67 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:11.422895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848137f7-ed2f-4530-8c7b-25e0fa780c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:11.423164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848137f7-ed2f-4530-8c7b-25e0fa780c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:11.425234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 937/1880] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:11.441668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb8276-291f-4d59-8a4d-a1998f16947c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:11.444455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-87fb8276-291f-4d59-8a4d-a1998f16947c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:11.454831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:11.515772 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848137f7-ed2f-4530-8c7b-25e0fa780c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:11.517349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848137f7-ed2f-4530-8c7b-25e0fa780c67 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:11.519001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb8276-291f-4d59-8a4d-a1998f16947c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:11.524443 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6306853-b927-4c72-ac6d-7400c62e4a2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:11.525439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6306853-b927-4c72-ac6d-7400c62e4a2a service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:11.525937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6306853-b927-4c72-ac6d-7400c62e4a2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:11.527880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6306853-b927-4c72-ac6d-7400c62e4a2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:11.528082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 944/1881] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:11.538566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848137f7-ed2f-4530-8c7b-25e0fa780c67 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:11.539998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-848137f7-ed2f-4530-8c7b-25e0fa780c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:11.543169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6306853-b927-4c72-ac6d-7400c62e4a2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:11.544800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6306853-b927-4c72-ac6d-7400c62e4a2a service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:11.553463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848137f7-ed2f-4530-8c7b-25e0fa780c67 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:11.557301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 938/1882] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:11.559394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:11.560398 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:11.560892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:11.561229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] PATH_INFO: `/v3/projects/aadd9ed106c246399fb408cf7c73ef04` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:11.563120 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aadd9ed106c246399fb408cf7c73ef04)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:11.565323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:11.653965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aadd9ed106c246399fb408cf7c73ef04', 'request_id': 'req-64b46876-0ac3-4414-aa9d-f35ec27ea47f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:11.667824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aadd9ed106c246399fb408cf7c73ef04', 'request_id': 'req-64b46876-0ac3-4414-aa9d-f35ec27ea47f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:11.688303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aadd9ed106c246399fb408cf7c73ef04', 'request_id': 'req-64b46876-0ac3-4414-aa9d-f35ec27ea47f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:11.706893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] The token cache is being invalidate because project aadd9ed106c246399fb408cf7c73ef04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:11.706893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:11.706893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64b46876-0ac3-4414-aa9d-f35ec27ea47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:11.706893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 945/1883] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:11 2022] DELETE /identity/v3/projects/aadd9ed106c246399fb408cf7c73ef04 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:11.713559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6b16e-0740-4090-b7b1-22f56ec7c4fb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:11.714842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6b16e-0740-4090-b7b1-22f56ec7c4fb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:11.715306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6b16e-0740-4090-b7b1-22f56ec7c4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:11.745650 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bfb6b16e-0740-4090-b7b1-22f56ec7c4fb None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:11.747244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bfb6b16e-0740-4090-b7b1-22f56ec7c4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:11.863086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 939/1884] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:11.871100 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae4273-7247-4bc9-bf1c-393cc95dd7ee None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:11.871394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae4273-7247-4bc9-bf1c-393cc95dd7ee None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:11.871663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae4273-7247-4bc9-bf1c-393cc95dd7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:11.901413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b3ae4273-7247-4bc9-bf1c-393cc95dd7ee None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:11.903515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b3ae4273-7247-4bc9-bf1c-393cc95dd7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:11.917041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 946/1885] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:11.918205 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6b16e-0740-4090-b7b1-22f56ec7c4fb None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:11.929755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6b16e-0740-4090-b7b1-22f56ec7c4fb None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:11.936844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dcd95-18df-45ef-a507-c9ba50b9ba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:11.937799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dcd95-18df-45ef-a507-c9ba50b9ba67 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:11.938123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dcd95-18df-45ef-a507-c9ba50b9ba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:11.938351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dcd95-18df-45ef-a507-c9ba50b9ba67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:11.939519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dcd95-18df-45ef-a507-c9ba50b9ba67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:11.939965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dcd95-18df-45ef-a507-c9ba50b9ba67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:11.941903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dcd95-18df-45ef-a507-c9ba50b9ba67 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:11.962836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 940/1886] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:11.968713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae4273-7247-4bc9-bf1c-393cc95dd7ee None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:11.983707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae4273-7247-4bc9-bf1c-393cc95dd7ee None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:11.991078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e123b-065f-4394-998f-067477ff31c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:11.992240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e123b-065f-4394-998f-067477ff31c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:11.992616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e123b-065f-4394-998f-067477ff31c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:11.992895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e123b-065f-4394-998f-067477ff31c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:11.993603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e123b-065f-4394-998f-067477ff31c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:11.995881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e123b-065f-4394-998f-067477ff31c3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.039039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 947/1887] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:11 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:12.040147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dcd95-18df-45ef-a507-c9ba50b9ba67 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.061864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dcd95-18df-45ef-a507-c9ba50b9ba67 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:12.076286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e123b-065f-4394-998f-067477ff31c3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.078082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-551e123b-065f-4394-998f-067477ff31c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:12.148769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:12.178080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e123b-065f-4394-998f-067477ff31c3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:12.189124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:12.191754 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.192050 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.192368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] PATH_INFO: `/v3/projects/c8ebad9e8fac4917bb1d8ec059fe847d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.196098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ebad9e8fac4917bb1d8ec059fe847d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:12.198095 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.212419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94bf7f-d20f-49f9-94ad-5f0d1e7d94d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:12.212757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:12.213548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94bf7f-d20f-49f9-94ad-5f0d1e7d94d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.214185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94bf7f-d20f-49f9-94ad-5f0d1e7d94d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.214468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94bf7f-d20f-49f9-94ad-5f0d1e7d94d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.215225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94bf7f-d20f-49f9-94ad-5f0d1e7d94d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:12.217802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94bf7f-d20f-49f9-94ad-5f0d1e7d94d2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.265589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 941/1888] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:12 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:12.266515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94bf7f-d20f-49f9-94ad-5f0d1e7d94d2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.268797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1f94bf7f-d20f-49f9-94ad-5f0d1e7d94d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:12.309674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ebad9e8fac4917bb1d8ec059fe847d', 'request_id': 'req-e1fbf508-a962-4466-9577-be8fa03bd1b5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.320226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ebad9e8fac4917bb1d8ec059fe847d', 'request_id': 'req-e1fbf508-a962-4466-9577-be8fa03bd1b5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.330527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ebad9e8fac4917bb1d8ec059fe847d', 'request_id': 'req-e1fbf508-a962-4466-9577-be8fa03bd1b5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.342997 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] The token cache is being invalidate because project c8ebad9e8fac4917bb1d8ec059fe847d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:12.343295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.343959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.346618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 948/1889] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:12 2022] DELETE /identity/v3/projects/c8ebad9e8fac4917bb1d8ec059fe847d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:12.347359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.348861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:12.352954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94bf7f-d20f-49f9-94ad-5f0d1e7d94d2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:12.419311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e12fc6-acc5-4c71-9a96-5017dac2bfcc tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec584808586544b89c9cd3493d76dede', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5dc1a2520b494a8e2b400faae58e28', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:12.420285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e12fc6-acc5-4c71-9a96-5017dac2bfcc tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.420677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e12fc6-acc5-4c71-9a96-5017dac2bfcc tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.421053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e12fc6-acc5-4c71-9a96-5017dac2bfcc tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.421877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbf508-a962-4466-9577-be8fa03bd1b5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:12.422004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e12fc6-acc5-4c71-9a96-5017dac2bfcc tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:12.422389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e12fc6-acc5-4c71-9a96-5017dac2bfcc tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:12.424159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e12fc6-acc5-4c71-9a96-5017dac2bfcc tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.427881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ef380-fddd-46c7-9dff-fee98daf1752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:12.428629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ef380-fddd-46c7-9dff-fee98daf1752 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.428980 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ef380-fddd-46c7-9dff-fee98daf1752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.429255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ef380-fddd-46c7-9dff-fee98daf1752 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.430018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ef380-fddd-46c7-9dff-fee98daf1752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:12.432055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ef380-fddd-46c7-9dff-fee98daf1752 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.437719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 942/1890] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:12.442855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e12fc6-acc5-4c71-9a96-5017dac2bfcc tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.447637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 949/1891] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:12.447773 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-01e12fc6-acc5-4c71-9a96-5017dac2bfcc tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:12.447865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ef380-fddd-46c7-9dff-fee98daf1752 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.449822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-605ef380-fddd-46c7-9dff-fee98daf1752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:12.461601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e12fc6-acc5-4c71-9a96-5017dac2bfcc tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:12.466225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:12.467116 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.467398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.467662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] PATH_INFO: `/v3/users/e46ec4c07a44456f8a23e7a918757e2c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.468584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ef380-fddd-46c7-9dff-fee98daf1752 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:12.468865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46ec4c07a44456f8a23e7a918757e2c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:12.470768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.473786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939eae1-2afb-44d3-8eff-fcd0d59132a9 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec584808586544b89c9cd3493d76dede', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5dc1a2520b494a8e2b400faae58e28', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:12.474457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939eae1-2afb-44d3-8eff-fcd0d59132a9 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.474742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939eae1-2afb-44d3-8eff-fcd0d59132a9 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.475029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939eae1-2afb-44d3-8eff-fcd0d59132a9 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.475826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939eae1-2afb-44d3-8eff-fcd0d59132a9 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:12.477883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939eae1-2afb-44d3-8eff-fcd0d59132a9 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.509648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 950/1892] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:12 2022] POST /identity/v3/projects => generated 354 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:12.509648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939eae1-2afb-44d3-8eff-fcd0d59132a9 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.525207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46ec4c07a44456f8a23e7a918757e2c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.525888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46ec4c07a44456f8a23e7a918757e2c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.533713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46ec4c07a44456f8a23e7a918757e2c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.535199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46ec4c07a44456f8a23e7a918757e2c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.537488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:12.552987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46ec4c07a44456f8a23e7a918757e2c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.569720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 943/1893] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:12 2022] DELETE /identity/v3/users/e46ec4c07a44456f8a23e7a918757e2c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:12.570905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.572436 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:12.592362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939eae1-2afb-44d3-8eff-fcd0d59132a9 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:12.607767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:12.673107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c3918-f973-4fcf-80fa-e456f682fcd6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:12.673107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02c35e-502c-4638-8732-5e2846e9735e tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec584808586544b89c9cd3493d76dede', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5dc1a2520b494a8e2b400faae58e28', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:12.673107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02c35e-502c-4638-8732-5e2846e9735e tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.673386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02c35e-502c-4638-8732-5e2846e9735e tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.673386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02c35e-502c-4638-8732-5e2846e9735e tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.673386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02c35e-502c-4638-8732-5e2846e9735e tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:12.673933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02c35e-502c-4638-8732-5e2846e9735e tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:12.675888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02c35e-502c-4638-8732-5e2846e9735e tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.680831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d955c-098d-4df1-8286-4bd713c4adf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:12.681522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d955c-098d-4df1-8286-4bd713c4adf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.682162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d955c-098d-4df1-8286-4bd713c4adf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.682435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d955c-098d-4df1-8286-4bd713c4adf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.683162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d955c-098d-4df1-8286-4bd713c4adf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:12.684979 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d955c-098d-4df1-8286-4bd713c4adf5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.688667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 944/1894] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:12.696670 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 951/1895] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:12.701304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02c35e-502c-4638-8732-5e2846e9735e tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.703376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9a02c35e-502c-4638-8732-5e2846e9735e tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:12.722922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02c35e-502c-4638-8732-5e2846e9735e tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:12.728927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9a9fc-8e37-4aab-8c23-79deddb8a976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:12.729842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9a9fc-8e37-4aab-8c23-79deddb8a976 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.730174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9a9fc-8e37-4aab-8c23-79deddb8a976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.730456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9a9fc-8e37-4aab-8c23-79deddb8a976 admin admin] PATH_INFO: `/v3/projects/b32c3ef2b04d498584152864a5c17e75/users/89e88f89daf94cba8b8483a15f938252/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.734755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d955c-098d-4df1-8286-4bd713c4adf5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.746616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d955c-098d-4df1-8286-4bd713c4adf5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:12.751034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec584808586544b89c9cd3493d76dede', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5dc1a2520b494a8e2b400faae58e28', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:12.751962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.752230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.752510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] PATH_INFO: `/v3/projects/49627af5a3dc4a35888d1dae009aa617` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.753105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9a9fc-8e37-4aab-8c23-79deddb8a976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32c3ef2b04d498584152864a5c17e75, user_id=89e88f89daf94cba8b8483a15f938252, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:12.756066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9a9fc-8e37-4aab-8c23-79deddb8a976 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.760323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: Authorizing `identity:delete_project(project_id=49627af5a3dc4a35888d1dae009aa617)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:12.762831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:12.772492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 945/1896] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:12 2022] PUT /identity/v3/projects/b32c3ef2b04d498584152864a5c17e75/users/89e88f89daf94cba8b8483a15f938252/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:12.782622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917f955-8897-428b-8ccc-0ecf6683516a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:12.782993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917f955-8897-428b-8ccc-0ecf6683516a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:12.783348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917f955-8897-428b-8ccc-0ecf6683516a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:12.812963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e917f955-8897-428b-8ccc-0ecf6683516a None None] MFA Rules not processed for user `89e88f89daf94cba8b8483a15f938252`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:12.814292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e917f955-8897-428b-8ccc-0ecf6683516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:12.834612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49627af5a3dc4a35888d1dae009aa617', 'request_id': 'req-861d286e-d6d0-4c04-8846-63fc5f1b38d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.843973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49627af5a3dc4a35888d1dae009aa617', 'request_id': 'req-861d286e-d6d0-4c04-8846-63fc5f1b38d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.853174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49627af5a3dc4a35888d1dae009aa617', 'request_id': 'req-861d286e-d6d0-4c04-8846-63fc5f1b38d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.865949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] The token cache is being invalidate because project 49627af5a3dc4a35888d1dae009aa617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:12.866234 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.866895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:12.868678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 952/1897] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:12 2022] DELETE /identity/v3/projects/49627af5a3dc4a35888d1dae009aa617 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:12.892983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.895889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:12.932150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 946/1898] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:12.935529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917f955-8897-428b-8ccc-0ecf6683516a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:12.940837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e917f955-8897-428b-8ccc-0ecf6683516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:12.987369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:13.005279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d286e-d6d0-4c04-8846-63fc5f1b38d3 tempest-VolumeQuotaClassesTest-149540419 tempest-VolumeQuotaClassesTest-149540419-project-admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.012143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.013811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.014104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.014541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] PATH_INFO: `/v3/projects/f4392c139639424a85a9ec046495ac11` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.015900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4392c139639424a85a9ec046495ac11)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.018915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.040347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917f955-8897-428b-8ccc-0ecf6683516a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.050868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97494f81-704b-4511-817c-74daf9985467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.052879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494f81-704b-4511-817c-74daf9985467 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.053277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494f81-704b-4511-817c-74daf9985467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.053940 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494f81-704b-4511-817c-74daf9985467 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.054989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97494f81-704b-4511-817c-74daf9985467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.060442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97494f81-704b-4511-817c-74daf9985467 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.082873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:13.099586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4392c139639424a85a9ec046495ac11', 'request_id': 'req-53786e2b-2c75-45c6-9975-97f7800dbe3e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:13.109455 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4392c139639424a85a9ec046495ac11', 'request_id': 'req-53786e2b-2c75-45c6-9975-97f7800dbe3e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:13.122137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4392c139639424a85a9ec046495ac11', 'request_id': 'req-53786e2b-2c75-45c6-9975-97f7800dbe3e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:13.125282 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 947/1899] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:12 2022] POST /identity/v3/projects => generated 363 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:13.130410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97494f81-704b-4511-817c-74daf9985467 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.143369 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] The token cache is being invalidate because project f4392c139639424a85a9ec046495ac11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:13.144352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:13.145416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53786e2b-2c75-45c6-9975-97f7800dbe3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:13.150814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 953/1900] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:12 2022] DELETE /identity/v3/projects/f4392c139639424a85a9ec046495ac11 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:13.161709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9067028-6e0b-4a26-98f7-31fd8eb5c37a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.162067 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9067028-6e0b-4a26-98f7-31fd8eb5c37a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.162426 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9067028-6e0b-4a26-98f7-31fd8eb5c37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.190290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a9067028-6e0b-4a26-98f7-31fd8eb5c37a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:13.191313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a9067028-6e0b-4a26-98f7-31fd8eb5c37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:13.210622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97494f81-704b-4511-817c-74daf9985467 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.217024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1e07a-26b1-4312-9a61-9328065b9f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.217948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1e07a-26b1-4312-9a61-9328065b9f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.218375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1e07a-26b1-4312-9a61-9328065b9f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.218792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1e07a-26b1-4312-9a61-9328065b9f44 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.219633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1e07a-26b1-4312-9a61-9328065b9f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.222282 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1e07a-26b1-4312-9a61-9328065b9f44 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.267943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 948/1901] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:13 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:13.269695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1e07a-26b1-4312-9a61-9328065b9f44 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.269900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 954/1902] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:13.271742 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-71a1e07a-26b1-4312-9a61-9328065b9f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:13.279479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfeb22b-face-4dda-85be-820637971e89 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.279770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfeb22b-face-4dda-85be-820637971e89 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.280263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfeb22b-face-4dda-85be-820637971e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.286971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1e07a-26b1-4312-9a61-9328065b9f44 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.293261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.294117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.294398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.294673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] PATH_INFO: `/v3/users/bf5fbc9446024ff78b4ff8fa0c6bd222` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.296346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf5fbc9446024ff78b4ff8fa0c6bd222)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.299491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.327001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-adfeb22b-face-4dda-85be-820637971e89 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:13.328426 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-adfeb22b-face-4dda-85be-820637971e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:13.358407 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 955/1903] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:13.362211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfeb22b-face-4dda-85be-820637971e89 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.366665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-adfeb22b-face-4dda-85be-820637971e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:13.385957 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf5fbc9446024ff78b4ff8fa0c6bd222'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:13.386644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf5fbc9446024ff78b4ff8fa0c6bd222'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:13.396162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf5fbc9446024ff78b4ff8fa0c6bd222'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:13.396642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfeb22b-face-4dda-85be-820637971e89 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.396869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf5fbc9446024ff78b4ff8fa0c6bd222'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:13.407140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13776c48-c434-4a5b-9edb-9a4e3e941a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.407845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13776c48-c434-4a5b-9edb-9a4e3e941a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.408156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13776c48-c434-4a5b-9edb-9a4e3e941a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.409720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13776c48-c434-4a5b-9edb-9a4e3e941a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.412015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf5fbc9446024ff78b4ff8fa0c6bd222'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:13.416446 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13776c48-c434-4a5b-9edb-9a4e3e941a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.417468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13776c48-c434-4a5b-9edb-9a4e3e941a7b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.456405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 956/1905] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:13.458649 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 949/1905] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:13 2022] DELETE /identity/v3/users/bf5fbc9446024ff78b4ff8fa0c6bd222 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:13.459513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13776c48-c434-4a5b-9edb-9a4e3e941a7b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.462844 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-13776c48-c434-4a5b-9edb-9a4e3e941a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:13.470366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.476757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:13.477893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13776c48-c434-4a5b-9edb-9a4e3e941a7b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.515166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:13.569251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef5694-db8f-4833-a9a9-0788bef46452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.569960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef5694-db8f-4833-a9a9-0788bef46452 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.570464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef5694-db8f-4833-a9a9-0788bef46452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.570926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef5694-db8f-4833-a9a9-0788bef46452 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.571833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef5694-db8f-4833-a9a9-0788bef46452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:13.572040 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef5694-db8f-4833-a9a9-0788bef46452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.574087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef5694-db8f-4833-a9a9-0788bef46452 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.594192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 957/1906] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:13.596771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dcd2e-c7f7-4158-bc05-5e7b10cd577b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.597274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dcd2e-c7f7-4158-bc05-5e7b10cd577b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.597637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dcd2e-c7f7-4158-bc05-5e7b10cd577b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.602490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41333b-88b9-484d-a8aa-eb910086b3e7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.621774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b6cb4-78d6-4a6e-ac58-f265c9fdf05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.622018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-516dcd2e-c7f7-4158-bc05-5e7b10cd577b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:13.622824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6cb4-78d6-4a6e-ac58-f265c9fdf05d admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.623388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6cb4-78d6-4a6e-ac58-f265c9fdf05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.623532 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-516dcd2e-c7f7-4158-bc05-5e7b10cd577b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:13.623683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6cb4-78d6-4a6e-ac58-f265c9fdf05d admin admin] PATH_INFO: `/v3/projects/518119da4a5c4df4a03c527344ed30cc/users/e38e482a21334a02bbf6ac2e78324550/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.640879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 958/1907] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:13.641722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516dcd2e-c7f7-4158-bc05-5e7b10cd577b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.649042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b6cb4-78d6-4a6e-ac58-f265c9fdf05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=518119da4a5c4df4a03c527344ed30cc, user_id=e38e482a21334a02bbf6ac2e78324550, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.654591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b6cb4-78d6-4a6e-ac58-f265c9fdf05d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.664672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516dcd2e-c7f7-4158-bc05-5e7b10cd577b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.672607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50614ca-1e09-419b-bf36-1a7488c8e801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.673617 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50614ca-1e09-419b-bf36-1a7488c8e801 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.675023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50614ca-1e09-419b-bf36-1a7488c8e801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.687908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 950/1908] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:13 2022] PUT /identity/v3/projects/518119da4a5c4df4a03c527344ed30cc/users/e38e482a21334a02bbf6ac2e78324550/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:13.688183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50614ca-1e09-419b-bf36-1a7488c8e801 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.691585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92262f6a-9478-4c3e-9b83-887c9ce8b358 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.695164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50614ca-1e09-419b-bf36-1a7488c8e801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.697327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50614ca-1e09-419b-bf36-1a7488c8e801 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.698306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92262f6a-9478-4c3e-9b83-887c9ce8b358 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.698756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92262f6a-9478-4c3e-9b83-887c9ce8b358 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.700939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 951/1909] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 20:58:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:58:13.701821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92262f6a-9478-4c3e-9b83-887c9ce8b358 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.704707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-92262f6a-9478-4c3e-9b83-887c9ce8b358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:13.750294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 959/1910] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:13 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:13.751459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50614ca-1e09-419b-bf36-1a7488c8e801 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.768770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:13.791401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92262f6a-9478-4c3e-9b83-887c9ce8b358 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.839093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50614ca-1e09-419b-bf36-1a7488c8e801 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.844238 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e6b21-7630-47b8-933b-32d6392e20a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.845062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e6b21-7630-47b8-933b-32d6392e20a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.845363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230b0d6-81f9-4d9a-b271-1c87fd7e7d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.845555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e6b21-7630-47b8-933b-32d6392e20a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.846027 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230b0d6-81f9-4d9a-b271-1c87fd7e7d9d service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.846137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e6b21-7630-47b8-933b-32d6392e20a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.846460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230b0d6-81f9-4d9a-b271-1c87fd7e7d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.846720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230b0d6-81f9-4d9a-b271-1c87fd7e7d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.847441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e6b21-7630-47b8-933b-32d6392e20a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.849170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e6b21-7630-47b8-933b-32d6392e20a2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.859727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 960/1911] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:13.861356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230b0d6-81f9-4d9a-b271-1c87fd7e7d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.861742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651da5c5-a4bf-48be-b92d-abc9d0869458 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.862148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651da5c5-a4bf-48be-b92d-abc9d0869458 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.862797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651da5c5-a4bf-48be-b92d-abc9d0869458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.863767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230b0d6-81f9-4d9a-b271-1c87fd7e7d9d service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.875074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 952/1912] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:13.876014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230b0d6-81f9-4d9a-b271-1c87fd7e7d9d service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.882526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-651da5c5-a4bf-48be-b92d-abc9d0869458 None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:13.883824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-651da5c5-a4bf-48be-b92d-abc9d0869458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:13.886966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230b0d6-81f9-4d9a-b271-1c87fd7e7d9d service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.893716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60eb4b-2f03-4f44-a490-53243e8b18ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.894880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60eb4b-2f03-4f44-a490-53243e8b18ab admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.895260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60eb4b-2f03-4f44-a490-53243e8b18ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.896136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60eb4b-2f03-4f44-a490-53243e8b18ab admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.896984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60eb4b-2f03-4f44-a490-53243e8b18ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.899112 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60eb4b-2f03-4f44-a490-53243e8b18ab admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.924619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 953/1913] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:13 2022] POST /identity/v3/users => generated 288 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:13.925672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60eb4b-2f03-4f44-a490-53243e8b18ab admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.937500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60eb4b-2f03-4f44-a490-53243e8b18ab admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:13.943359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672646a-8198-4ee5-8ca0-5f4e70525bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:13.944320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672646a-8198-4ee5-8ca0-5f4e70525bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.944604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672646a-8198-4ee5-8ca0-5f4e70525bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.944882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672646a-8198-4ee5-8ca0-5f4e70525bd9 admin admin] PATH_INFO: `/v3/projects/518119da4a5c4df4a03c527344ed30cc/users/e38e482a21334a02bbf6ac2e78324550/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:13.947202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672646a-8198-4ee5-8ca0-5f4e70525bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518119da4a5c4df4a03c527344ed30cc, user_id=e38e482a21334a02bbf6ac2e78324550, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:13.949957 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672646a-8198-4ee5-8ca0-5f4e70525bd9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:13.968235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 954/1914] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:13 2022] PUT /identity/v3/projects/518119da4a5c4df4a03c527344ed30cc/users/e38e482a21334a02bbf6ac2e78324550/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:13.968235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672646a-8198-4ee5-8ca0-5f4e70525bd9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:13.974674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 961/1915] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:13.981804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49407c-c27e-48d3-8acc-c519ece42bc6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:13.982164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49407c-c27e-48d3-8acc-c519ece42bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:13.982714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49407c-c27e-48d3-8acc-c519ece42bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.004677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:14.013711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ca49407c-c27e-48d3-8acc-c519ece42bc6 None None] MFA Rules not processed for user `e38e482a21334a02bbf6ac2e78324550`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:14.015185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ca49407c-c27e-48d3-8acc-c519ece42bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:14.045787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672646a-8198-4ee5-8ca0-5f4e70525bd9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:14.050955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e164e-16f4-40ca-9f76-afd8d09cfc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:14.051670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e164e-16f4-40ca-9f76-afd8d09cfc7b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.052038 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e164e-16f4-40ca-9f76-afd8d09cfc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.052361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e164e-16f4-40ca-9f76-afd8d09cfc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.053098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e164e-16f4-40ca-9f76-afd8d09cfc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:14.055737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e164e-16f4-40ca-9f76-afd8d09cfc7b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:14.066550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 955/1916] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:14.067515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e164e-16f4-40ca-9f76-afd8d09cfc7b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:14.069448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7e6e164e-16f4-40ca-9f76-afd8d09cfc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:14.132908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 962/1917] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:14.134069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49407c-c27e-48d3-8acc-c519ece42bc6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:14.145338 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49407c-c27e-48d3-8acc-c519ece42bc6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:14.151151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8ce0b-fd01-4422-a2ed-94b85430f64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:14.151876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8ce0b-fd01-4422-a2ed-94b85430f64c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.152272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8ce0b-fd01-4422-a2ed-94b85430f64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.152616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8ce0b-fd01-4422-a2ed-94b85430f64c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.153433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8ce0b-fd01-4422-a2ed-94b85430f64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:14.155143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8ce0b-fd01-4422-a2ed-94b85430f64c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:14.167382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 963/1918] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:14.168398 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8ce0b-fd01-4422-a2ed-94b85430f64c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:14.171214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f7e8ce0b-fd01-4422-a2ed-94b85430f64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:14.172018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e164e-16f4-40ca-9f76-afd8d09cfc7b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:14.184978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86c4bb-f67a-482f-a759-f672ae66a686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:14.185770 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86c4bb-f67a-482f-a759-f672ae66a686 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.186055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86c4bb-f67a-482f-a759-f672ae66a686 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.186333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86c4bb-f67a-482f-a759-f672ae66a686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.193616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:14.201534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86c4bb-f67a-482f-a759-f672ae66a686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:14.204419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86c4bb-f67a-482f-a759-f672ae66a686 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:14.218578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 956/1919] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:14.219802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86c4bb-f67a-482f-a759-f672ae66a686 service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:14.232383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86c4bb-f67a-482f-a759-f672ae66a686 service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:14.238721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67f2cd-b677-4f5c-8b0e-f34c9c8547d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:14.239485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67f2cd-b677-4f5c-8b0e-f34c9c8547d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.239771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67f2cd-b677-4f5c-8b0e-f34c9c8547d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.240048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67f2cd-b677-4f5c-8b0e-f34c9c8547d3 admin admin] PATH_INFO: `/v3/projects/feb2f85fc9d0484f827d5be0d58a3665/users/a443cab4d3334350a1f0ade50e9af075/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.261598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8ce0b-fd01-4422-a2ed-94b85430f64c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:14.267088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753791a5-8fdb-4602-bfbb-892c69db5b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:14.268434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753791a5-8fdb-4602-bfbb-892c69db5b88 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.269272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753791a5-8fdb-4602-bfbb-892c69db5b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.269647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753791a5-8fdb-4602-bfbb-892c69db5b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.274820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67f2cd-b677-4f5c-8b0e-f34c9c8547d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb2f85fc9d0484f827d5be0d58a3665, user_id=a443cab4d3334350a1f0ade50e9af075, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:14.277703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67f2cd-b677-4f5c-8b0e-f34c9c8547d3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:14.281743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753791a5-8fdb-4602-bfbb-892c69db5b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:14.283866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753791a5-8fdb-4602-bfbb-892c69db5b88 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:14.297615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 964/1920] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:14 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:14.301618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 957/1921] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:14 2022] PUT /identity/v3/projects/feb2f85fc9d0484f827d5be0d58a3665/users/a443cab4d3334350a1f0ade50e9af075/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:14.314028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224873c2-d822-41a3-a1b8-7a241ca2986b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.314246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224873c2-d822-41a3-a1b8-7a241ca2986b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.314430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224873c2-d822-41a3-a1b8-7a241ca2986b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.359895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-224873c2-d822-41a3-a1b8-7a241ca2986b None None] MFA Rules not processed for user `a443cab4d3334350a1f0ade50e9af075`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:14.361381 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-224873c2-d822-41a3-a1b8-7a241ca2986b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:14.410427 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67f2cd-b677-4f5c-8b0e-f34c9c8547d3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:14.412364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7d67f2cd-b677-4f5c-8b0e-f34c9c8547d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:14.463559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 965/1922] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:14.464552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224873c2-d822-41a3-a1b8-7a241ca2986b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:14.519613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67f2cd-b677-4f5c-8b0e-f34c9c8547d3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:14.525837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:14.526975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.527324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.528301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] PATH_INFO: `/v3/projects/bb75a0a70790432ca2cab13c875e2876` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.531144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb75a0a70790432ca2cab13c875e2876)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:14.533281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:14.535833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224873c2-d822-41a3-a1b8-7a241ca2986b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:14.540674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021e005-9748-4b59-9d30-4264bee2b456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:14.541089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021e005-9748-4b59-9d30-4264bee2b456 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.541462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021e005-9748-4b59-9d30-4264bee2b456 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.541824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021e005-9748-4b59-9d30-4264bee2b456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.544843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6021e005-9748-4b59-9d30-4264bee2b456 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:14.579803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:14.595886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb75a0a70790432ca2cab13c875e2876', 'request_id': 'req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:14.605118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb75a0a70790432ca2cab13c875e2876', 'request_id': 'req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:14.613507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021e005-9748-4b59-9d30-4264bee2b456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:14.614373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb75a0a70790432ca2cab13c875e2876', 'request_id': 'req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:14.615168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021e005-9748-4b59-9d30-4264bee2b456 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:14.625419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 966/1923] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:14.626210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021e005-9748-4b59-9d30-4264bee2b456 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:14.627036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] The token cache is being invalidate because project bb75a0a70790432ca2cab13c875e2876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:14.627423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:14.628480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:14.632582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 958/1924] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:14 2022] DELETE /identity/v3/projects/bb75a0a70790432ca2cab13c875e2876 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:14.632675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:14.634454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:14.636518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:14.639876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021e005-9748-4b59-9d30-4264bee2b456 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:14.722396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ea072-ad8c-475f-96eb-1cbce10ecec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:14.723223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ea072-ad8c-475f-96eb-1cbce10ecec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.723505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ea072-ad8c-475f-96eb-1cbce10ecec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.723776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ea072-ad8c-475f-96eb-1cbce10ecec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.724509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ea072-ad8c-475f-96eb-1cbce10ecec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:14.725214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b4b2d-6110-42dd-90e2-577edceb5cd2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:14.726436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ea072-ad8c-475f-96eb-1cbce10ecec9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:14.730794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:14.731414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.732016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.732223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] PATH_INFO: `/v3/users/ec584808586544b89c9cd3493d76dede` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.733651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec584808586544b89c9cd3493d76dede)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:14.737756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:14.795843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 967/1925] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:14 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:14.800364 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ea072-ad8c-475f-96eb-1cbce10ecec9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:14.802568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa2ea072-ad8c-475f-96eb-1cbce10ecec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:14.846394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec584808586544b89c9cd3493d76dede'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:14.847156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec584808586544b89c9cd3493d76dede'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:14.855832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec584808586544b89c9cd3493d76dede'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:14.856295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec584808586544b89c9cd3493d76dede'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:14.870514 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec584808586544b89c9cd3493d76dede'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:14.886848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 959/1926] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:14 2022] DELETE /identity/v3/users/ec584808586544b89c9cd3493d76dede => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:14.888688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ea072-ad8c-475f-96eb-1cbce10ecec9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:14.971375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f4833-07de-4f2e-8043-d99b7357de8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:14.972235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f4833-07de-4f2e-8043-d99b7357de8a admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:14.972583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f4833-07de-4f2e-8043-d99b7357de8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:14.972944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f4833-07de-4f2e-8043-d99b7357de8a admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:14.973694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f4833-07de-4f2e-8043-d99b7357de8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:14.975698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f4833-07de-4f2e-8043-d99b7357de8a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:14.997152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 968/1927] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:14 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:15.001226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.002693 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:15.016391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7acff-021e-4b06-a44d-0bf29dc3dc38 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.021258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38990cfc-196e-4c95-85fa-b2628eb33d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.021949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38990cfc-196e-4c95-85fa-b2628eb33d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.022232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38990cfc-196e-4c95-85fa-b2628eb33d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.022509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38990cfc-196e-4c95-85fa-b2628eb33d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.023299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38990cfc-196e-4c95-85fa-b2628eb33d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.024971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38990cfc-196e-4c95-85fa-b2628eb33d41 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.041068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 960/1928] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:15.042507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f4833-07de-4f2e-8043-d99b7357de8a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.056047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f4833-07de-4f2e-8043-d99b7357de8a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.061689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c486b62-cecf-462a-924c-e49a7f76cedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.062624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c486b62-cecf-462a-924c-e49a7f76cedb admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.062905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c486b62-cecf-462a-924c-e49a7f76cedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.063157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c486b62-cecf-462a-924c-e49a7f76cedb admin admin] PATH_INFO: `/v3/projects/170af42ec61f4dbbbf6ba9606848001d/users/7e1efafdd5be4237beec9bcab2fb2045/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.079585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c486b62-cecf-462a-924c-e49a7f76cedb admin admin] RBAC: Authorizing `identity:create_grant(project_id=170af42ec61f4dbbbf6ba9606848001d, user_id=7e1efafdd5be4237beec9bcab2fb2045, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.082838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c486b62-cecf-462a-924c-e49a7f76cedb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.093460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:15.099785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 969/1929] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:15 2022] PUT /identity/v3/projects/170af42ec61f4dbbbf6ba9606848001d/users/7e1efafdd5be4237beec9bcab2fb2045/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:15.107572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38990cfc-196e-4c95-85fa-b2628eb33d41 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.119540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38990cfc-196e-4c95-85fa-b2628eb33d41 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.146448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:15.192097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b417d-a7cd-47e0-b64d-41511e654d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.192815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b417d-a7cd-47e0-b64d-41511e654d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.193098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b417d-a7cd-47e0-b64d-41511e654d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.193375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b417d-a7cd-47e0-b64d-41511e654d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.194124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b417d-a7cd-47e0-b64d-41511e654d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.195771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b417d-a7cd-47e0-b64d-41511e654d57 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.205890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 961/1930] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:15.210278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c486b62-cecf-462a-924c-e49a7f76cedb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.221184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c486b62-cecf-462a-924c-e49a7f76cedb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.226320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed051eee-6fee-46d0-9605-83aa4ea7e9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.227261 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed051eee-6fee-46d0-9605-83aa4ea7e9fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.227542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed051eee-6fee-46d0-9605-83aa4ea7e9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.227811 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed051eee-6fee-46d0-9605-83aa4ea7e9fd admin admin] PATH_INFO: `/v3/projects/170af42ec61f4dbbbf6ba9606848001d/users/7e1efafdd5be4237beec9bcab2fb2045/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.229807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed051eee-6fee-46d0-9605-83aa4ea7e9fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=170af42ec61f4dbbbf6ba9606848001d, user_id=7e1efafdd5be4237beec9bcab2fb2045, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.232577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed051eee-6fee-46d0-9605-83aa4ea7e9fd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.243791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 970/1931] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:15 2022] PUT /identity/v3/projects/170af42ec61f4dbbbf6ba9606848001d/users/7e1efafdd5be4237beec9bcab2fb2045/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:15.254585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1477a-315a-4e66-9928-8ba11fc6bb66 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.254918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1477a-315a-4e66-9928-8ba11fc6bb66 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.255218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1477a-315a-4e66-9928-8ba11fc6bb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.264540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed051eee-6fee-46d0-9605-83aa4ea7e9fd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.266438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ed051eee-6fee-46d0-9605-83aa4ea7e9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:15.285586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cfe1477a-315a-4e66-9928-8ba11fc6bb66 None None] MFA Rules not processed for user `7e1efafdd5be4237beec9bcab2fb2045`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:15.286932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cfe1477a-315a-4e66-9928-8ba11fc6bb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:15.353312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed051eee-6fee-46d0-9605-83aa4ea7e9fd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.358453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.359393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.359681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.359958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] PATH_INFO: `/v3/projects/8d5dc1a2520b494a8e2b400faae58e28` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.361233 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d5dc1a2520b494a8e2b400faae58e28)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.363920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.369990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 962/1932] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:15.382626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1477a-315a-4e66-9928-8ba11fc6bb66 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.385179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cfe1477a-315a-4e66-9928-8ba11fc6bb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:15.422258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d5dc1a2520b494a8e2b400faae58e28', 'request_id': 'req-4f9efcfc-d2df-404f-9554-b23a57ef7232'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:15.430076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d5dc1a2520b494a8e2b400faae58e28', 'request_id': 'req-4f9efcfc-d2df-404f-9554-b23a57ef7232'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:15.438820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d5dc1a2520b494a8e2b400faae58e28', 'request_id': 'req-4f9efcfc-d2df-404f-9554-b23a57ef7232'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:15.452145 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] The token cache is being invalidate because project 8d5dc1a2520b494a8e2b400faae58e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:15.452610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:15.453468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4f9efcfc-d2df-404f-9554-b23a57ef7232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:15.454513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1477a-315a-4e66-9928-8ba11fc6bb66 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.455641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 971/1933] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:15 2022] DELETE /identity/v3/projects/8d5dc1a2520b494a8e2b400faae58e28 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:15.460478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc956e7-9bde-425f-aaad-e500314ae4a0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.460909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc956e7-9bde-425f-aaad-e500314ae4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.461285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc956e7-9bde-425f-aaad-e500314ae4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.480306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6dc956e7-9bde-425f-aaad-e500314ae4a0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:15.481626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6dc956e7-9bde-425f-aaad-e500314ae4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:15.515036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad4b59-307f-48d2-a348-2f4286ab8e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.516337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad4b59-307f-48d2-a348-2f4286ab8e83 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.516767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad4b59-307f-48d2-a348-2f4286ab8e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.517190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad4b59-307f-48d2-a348-2f4286ab8e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.528029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-43ad4b59-307f-48d2-a348-2f4286ab8e83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:15.539214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:15.545738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 972/1934] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:15.554656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a38b8-607c-4749-954f-58930723d5d2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.555038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a38b8-607c-4749-954f-58930723d5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.555377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a38b8-607c-4749-954f-58930723d5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.573919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c59a38b8-607c-4749-954f-58930723d5d2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:15.575218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c59a38b8-607c-4749-954f-58930723d5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:15.583998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 973/1935] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:15.585421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad4b59-307f-48d2-a348-2f4286ab8e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.586927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad4b59-307f-48d2-a348-2f4286ab8e83 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.587907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a38b8-607c-4749-954f-58930723d5d2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.597022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 963/1936] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:15.597167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a38b8-607c-4749-954f-58930723d5d2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.603049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d5f85-455e-45da-a784-ff28d94f9423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.603760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d5f85-455e-45da-a784-ff28d94f9423 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.603958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d5f85-455e-45da-a784-ff28d94f9423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.604334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d5f85-455e-45da-a784-ff28d94f9423 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.605325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d5f85-455e-45da-a784-ff28d94f9423 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:15.605743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d5f85-455e-45da-a784-ff28d94f9423 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.607414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d5f85-455e-45da-a784-ff28d94f9423 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.615090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:15.621670 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 974/1937] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:58:15.626195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad4b59-307f-48d2-a348-2f4286ab8e83 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.638795 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad4b59-307f-48d2-a348-2f4286ab8e83 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.645131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916f557-5e41-4a34-9bde-a245a2b768f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.645829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f557-5e41-4a34-9bde-a245a2b768f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.646109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f557-5e41-4a34-9bde-a245a2b768f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.646383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f557-5e41-4a34-9bde-a245a2b768f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.647096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916f557-5e41-4a34-9bde-a245a2b768f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.649178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916f557-5e41-4a34-9bde-a245a2b768f9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.682932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 964/1938] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:15 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:15.686093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d5f85-455e-45da-a784-ff28d94f9423 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.697249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d5f85-455e-45da-a784-ff28d94f9423 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.764968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b193f-f77c-40a8-9d4c-4475c3a19330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.765919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b193f-f77c-40a8-9d4c-4475c3a19330 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.766076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b193f-f77c-40a8-9d4c-4475c3a19330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.766332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b193f-f77c-40a8-9d4c-4475c3a19330 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.767048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b193f-f77c-40a8-9d4c-4475c3a19330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.769395 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b193f-f77c-40a8-9d4c-4475c3a19330 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.795625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 975/1939] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:15 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:15.798290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916f557-5e41-4a34-9bde-a245a2b768f9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.815482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916f557-5e41-4a34-9bde-a245a2b768f9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.822207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3259fec-83c9-4c64-a7b4-c2779aae1e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.823097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3259fec-83c9-4c64-a7b4-c2779aae1e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.823665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3259fec-83c9-4c64-a7b4-c2779aae1e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.824010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3259fec-83c9-4c64-a7b4-c2779aae1e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.824949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3259fec-83c9-4c64-a7b4-c2779aae1e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.826582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3259fec-83c9-4c64-a7b4-c2779aae1e25 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.852168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 965/1940] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:15.853501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b193f-f77c-40a8-9d4c-4475c3a19330 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.866038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b193f-f77c-40a8-9d4c-4475c3a19330 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.872149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376623e8-05ea-4633-a4c4-2b904b78e6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.872874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376623e8-05ea-4633-a4c4-2b904b78e6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.873158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376623e8-05ea-4633-a4c4-2b904b78e6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.873464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376623e8-05ea-4633-a4c4-2b904b78e6b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.874205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376623e8-05ea-4633-a4c4-2b904b78e6b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.876072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376623e8-05ea-4633-a4c4-2b904b78e6b3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.894596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 976/1941] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:15.897949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3259fec-83c9-4c64-a7b4-c2779aae1e25 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:15.910087 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3259fec-83c9-4c64-a7b4-c2779aae1e25 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:15.915018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2a63f-952d-42cb-a8c4-cb7900d3584c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:15.916506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2a63f-952d-42cb-a8c4-cb7900d3584c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.916758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2a63f-952d-42cb-a8c4-cb7900d3584c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.917361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2a63f-952d-42cb-a8c4-cb7900d3584c admin admin] PATH_INFO: `/v3/projects/61a67646f4ef4d768d656bc1be25adc2/users/fc6b58cfc329475d9700fedf909884d3/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:15.936129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2a63f-952d-42cb-a8c4-cb7900d3584c admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a67646f4ef4d768d656bc1be25adc2, user_id=fc6b58cfc329475d9700fedf909884d3, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:15.939280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2a63f-952d-42cb-a8c4-cb7900d3584c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:15.956843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 966/1942] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:15 2022] PUT /identity/v3/projects/61a67646f4ef4d768d656bc1be25adc2/users/fc6b58cfc329475d9700fedf909884d3/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:15.968492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97732d-9aa8-4bcc-8a2c-07735aa921ca None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:15.968785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97732d-9aa8-4bcc-8a2c-07735aa921ca None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:15.969090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97732d-9aa8-4bcc-8a2c-07735aa921ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:16.000645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1e97732d-9aa8-4bcc-8a2c-07735aa921ca None None] MFA Rules not processed for user `fc6b58cfc329475d9700fedf909884d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:16.002035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1e97732d-9aa8-4bcc-8a2c-07735aa921ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:16.071294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 977/1943] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:16.075928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2a63f-952d-42cb-a8c4-cb7900d3584c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:16.140323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2a63f-952d-42cb-a8c4-cb7900d3584c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:16.146057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353f3ed-cc15-414d-a09e-27b9e47fef52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:16.146720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f3ed-cc15-414d-a09e-27b9e47fef52 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:16.147001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f3ed-cc15-414d-a09e-27b9e47fef52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:16.147299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f3ed-cc15-414d-a09e-27b9e47fef52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:16.147992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353f3ed-cc15-414d-a09e-27b9e47fef52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:16.150006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353f3ed-cc15-414d-a09e-27b9e47fef52 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:16.176894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 967/1944] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:16 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:16.181200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97732d-9aa8-4bcc-8a2c-07735aa921ca None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:16.192781 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97732d-9aa8-4bcc-8a2c-07735aa921ca None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:16.251484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9caa4-74b5-440e-9c2e-142f212ab3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:16.252365 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9caa4-74b5-440e-9c2e-142f212ab3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:16.252707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9caa4-74b5-440e-9c2e-142f212ab3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:16.253241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9caa4-74b5-440e-9c2e-142f212ab3c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:16.253690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9caa4-74b5-440e-9c2e-142f212ab3c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:16.255823 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9caa4-74b5-440e-9c2e-142f212ab3c8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:16.267787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:16.280449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 978/1945] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:16 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:16.284266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353f3ed-cc15-414d-a09e-27b9e47fef52 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:16.295111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353f3ed-cc15-414d-a09e-27b9e47fef52 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:16.300515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271877a9-a28b-4397-84d0-6d30ade55c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:16.301189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271877a9-a28b-4397-84d0-6d30ade55c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:16.301465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271877a9-a28b-4397-84d0-6d30ade55c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:16.301733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271877a9-a28b-4397-84d0-6d30ade55c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:16.302465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271877a9-a28b-4397-84d0-6d30ade55c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:16.304068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271877a9-a28b-4397-84d0-6d30ade55c5a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:16.317790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 968/1946] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:16.322588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9caa4-74b5-440e-9c2e-142f212ab3c8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:16.333043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9caa4-74b5-440e-9c2e-142f212ab3c8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:16.338564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aff3c9-f343-4c06-839e-4853aefaac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:16.339965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aff3c9-f343-4c06-839e-4853aefaac04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:16.340255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aff3c9-f343-4c06-839e-4853aefaac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:16.340572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aff3c9-f343-4c06-839e-4853aefaac04 admin admin] PATH_INFO: `/v3/projects/abb1c4c127864f8db7bb2be0880141d7/users/c722117ea9504830b31e25dd3734d7a8/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:16.357264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aff3c9-f343-4c06-839e-4853aefaac04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb1c4c127864f8db7bb2be0880141d7, user_id=c722117ea9504830b31e25dd3734d7a8, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:16.360245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aff3c9-f343-4c06-839e-4853aefaac04 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:16.376858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 979/1947] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:16 2022] PUT /identity/v3/projects/abb1c4c127864f8db7bb2be0880141d7/users/c722117ea9504830b31e25dd3734d7a8/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:16.381210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271877a9-a28b-4397-84d0-6d30ade55c5a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:16.391697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271877a9-a28b-4397-84d0-6d30ade55c5a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:16.435124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:16.464056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4351a1d-744c-431f-9ed6-0fe457129796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:16.464784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4351a1d-744c-431f-9ed6-0fe457129796 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:16.465062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4351a1d-744c-431f-9ed6-0fe457129796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:16.465336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4351a1d-744c-431f-9ed6-0fe457129796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:16.466050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4351a1d-744c-431f-9ed6-0fe457129796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:16.467653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4351a1d-744c-431f-9ed6-0fe457129796 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:16.477377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 969/1948] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:16.483350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aff3c9-f343-4c06-839e-4853aefaac04 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:16.495265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aff3c9-f343-4c06-839e-4853aefaac04 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:16.500675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4476f7a-4dc5-4d2f-aca2-9195acf7b240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:16.501739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4476f7a-4dc5-4d2f-aca2-9195acf7b240 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:16.502020 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4476f7a-4dc5-4d2f-aca2-9195acf7b240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:16.502296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4476f7a-4dc5-4d2f-aca2-9195acf7b240 admin admin] PATH_INFO: `/v3/projects/abb1c4c127864f8db7bb2be0880141d7/users/c722117ea9504830b31e25dd3734d7a8/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:16.504704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4476f7a-4dc5-4d2f-aca2-9195acf7b240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb1c4c127864f8db7bb2be0880141d7, user_id=c722117ea9504830b31e25dd3734d7a8, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:16.507412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4476f7a-4dc5-4d2f-aca2-9195acf7b240 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:16.519055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 980/1949] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:16 2022] PUT /identity/v3/projects/abb1c4c127864f8db7bb2be0880141d7/users/c722117ea9504830b31e25dd3734d7a8/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:16.529260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1250-af2b-4217-8786-317cddb4d4df None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:16.529911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1250-af2b-4217-8786-317cddb4d4df None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:16.530198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1250-af2b-4217-8786-317cddb4d4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:16.559831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4bca1250-af2b-4217-8786-317cddb4d4df None None] MFA Rules not processed for user `c722117ea9504830b31e25dd3734d7a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:16.561117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4bca1250-af2b-4217-8786-317cddb4d4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:16.632492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 970/1950] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:16.647193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4476f7a-4dc5-4d2f-aca2-9195acf7b240 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:16.648930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e4476f7a-4dc5-4d2f-aca2-9195acf7b240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:16.707295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4476f7a-4dc5-4d2f-aca2-9195acf7b240 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:16.711074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f8f2f-6402-4a3c-8a72-b56ef6dcf54d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:16.711731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f8f2f-6402-4a3c-8a72-b56ef6dcf54d service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:16.712001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f8f2f-6402-4a3c-8a72-b56ef6dcf54d service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:16.712305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f8f2f-6402-4a3c-8a72-b56ef6dcf54d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:16.721356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f8f2f-6402-4a3c-8a72-b56ef6dcf54d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:16.722797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f8f2f-6402-4a3c-8a72-b56ef6dcf54d service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:16.732122 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 981/1951] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:16.834517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca1250-af2b-4217-8786-317cddb4d4df None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:16.836218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4bca1250-af2b-4217-8786-317cddb4d4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:16.850764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca1250-af2b-4217-8786-317cddb4d4df None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:16.854612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25359995-73af-4077-95ad-4e5dcbb47e33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:16.855272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25359995-73af-4077-95ad-4e5dcbb47e33 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:16.855554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25359995-73af-4077-95ad-4e5dcbb47e33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:16.856351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25359995-73af-4077-95ad-4e5dcbb47e33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:16.925292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25359995-73af-4077-95ad-4e5dcbb47e33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:16.926857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25359995-73af-4077-95ad-4e5dcbb47e33 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:16.937918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 971/1952] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:19.488245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f8f2f-6402-4a3c-8a72-b56ef6dcf54d service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:19.490862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-443f8f2f-6402-4a3c-8a72-b56ef6dcf54d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:19.579928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f8f2f-6402-4a3c-8a72-b56ef6dcf54d service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:19.586518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:19.590243 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:19.590672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:19.590956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] PATH_INFO: `/v3/users/89e88f89daf94cba8b8483a15f938252` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:19.593055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89e88f89daf94cba8b8483a15f938252)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:19.595042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:19.672702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89e88f89daf94cba8b8483a15f938252'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:19.688762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89e88f89daf94cba8b8483a15f938252'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:19.691206 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89e88f89daf94cba8b8483a15f938252'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:19.702211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89e88f89daf94cba8b8483a15f938252'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:19.704321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:19.721585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89e88f89daf94cba8b8483a15f938252'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:19.729906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 982/1953] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:19 2022] DELETE /identity/v3/users/89e88f89daf94cba8b8483a15f938252 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:19.734363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ee882-bfde-441d-adc4-57f12441b666 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:19.745244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ee882-bfde-441d-adc4-57f12441b666 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:19.745810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ee882-bfde-441d-adc4-57f12441b666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:19.794987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-891ee882-bfde-441d-adc4-57f12441b666 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:19.797299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-891ee882-bfde-441d-adc4-57f12441b666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:19.916864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:19.942294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 972/1954] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:19.991763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:19.993485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:20.076445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a81f20-c708-4594-a99e-77b202394b84 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:20.086024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a78e9-ceee-49a4-9292-365af7792ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:20.086697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a78e9-ceee-49a4-9292-365af7792ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:20.086974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a78e9-ceee-49a4-9292-365af7792ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:20.087258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a78e9-ceee-49a4-9292-365af7792ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:20.098022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a78e9-ceee-49a4-9292-365af7792ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:20.099479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a78e9-ceee-49a4-9292-365af7792ca2 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:20.110989 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 983/1955] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:20.566104 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ee882-bfde-441d-adc4-57f12441b666 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:20.568052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-891ee882-bfde-441d-adc4-57f12441b666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:20.582952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ee882-bfde-441d-adc4-57f12441b666 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:20.588772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:20.589724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:20.590011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:20.590287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] PATH_INFO: `/v3/projects/b32c3ef2b04d498584152864a5c17e75` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:20.591485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32c3ef2b04d498584152864a5c17e75)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:20.593472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:20.658298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32c3ef2b04d498584152864a5c17e75', 'request_id': 'req-b62b2bb3-41d8-47d1-a723-ce4178b8e220'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.665358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32c3ef2b04d498584152864a5c17e75', 'request_id': 'req-b62b2bb3-41d8-47d1-a723-ce4178b8e220'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.670247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a78e9-ceee-49a4-9292-365af7792ca2 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:20.672010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-191a78e9-ceee-49a4-9292-365af7792ca2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:20.677272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32c3ef2b04d498584152864a5c17e75', 'request_id': 'req-b62b2bb3-41d8-47d1-a723-ce4178b8e220'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.686821 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a78e9-ceee-49a4-9292-365af7792ca2 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:20.690728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] The token cache is being invalidate because project b32c3ef2b04d498584152864a5c17e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:20.691396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.692278 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.696582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 973/1956] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:20 2022] DELETE /identity/v3/projects/b32c3ef2b04d498584152864a5c17e75 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:20.701926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:20.704723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:20.757752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:20.758751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:20.759023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:20.759341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] PATH_INFO: `/v3/users/a443cab4d3334350a1f0ade50e9af075` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:20.760651 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a443cab4d3334350a1f0ade50e9af075)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:20.762600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:20.793516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b2bb3-41d8-47d1-a723-ce4178b8e220 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:20.799026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:20.799963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:20.800280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:20.800595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] PATH_INFO: `/v3/users/e38e482a21334a02bbf6ac2e78324550` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:20.801793 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] RBAC: Authorizing `identity:delete_user(user_id=e38e482a21334a02bbf6ac2e78324550)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:20.803743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:20.824382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a443cab4d3334350a1f0ade50e9af075'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.832420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a443cab4d3334350a1f0ade50e9af075'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.833505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a443cab4d3334350a1f0ade50e9af075'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.843176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a443cab4d3334350a1f0ade50e9af075'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.860264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a443cab4d3334350a1f0ade50e9af075'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.864762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e38e482a21334a02bbf6ac2e78324550'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.867666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 984/1957] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:20 2022] DELETE /identity/v3/users/a443cab4d3334350a1f0ade50e9af075 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:20.868859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:20.870301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:20.872281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e38e482a21334a02bbf6ac2e78324550'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.875962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:20.883842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e38e482a21334a02bbf6ac2e78324550'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.884696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e38e482a21334a02bbf6ac2e78324550'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.897581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6811568-25a9-4f8c-92af-908f510290df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e38e482a21334a02bbf6ac2e78324550'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:20.907047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:20.913299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 974/1958] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:20 2022] DELETE /identity/v3/users/e38e482a21334a02bbf6ac2e78324550 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:20.914711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b43366-048c-42c7-ade8-503edaa3fe6b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:20.915008 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b43366-048c-42c7-ade8-503edaa3fe6b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:20.915291 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b43366-048c-42c7-ade8-503edaa3fe6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:20.940555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d0b43366-048c-42c7-ade8-503edaa3fe6b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:20.942151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d0b43366-048c-42c7-ade8-503edaa3fe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:20.962453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836015e8-226f-4503-af32-a45d29ce8c2c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:21.034377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 975/1959] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:21.041682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b43366-048c-42c7-ade8-503edaa3fe6b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:21.043609 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d0b43366-048c-42c7-ade8-503edaa3fe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:21.063751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:21.064835 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.065816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.066093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] PATH_INFO: `/v3/users/fc6b58cfc329475d9700fedf909884d3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.068784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc6b58cfc329475d9700fedf909884d3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:21.071018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:21.113256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b43366-048c-42c7-ade8-503edaa3fe6b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:21.118116 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158914aa-2e6e-4b62-8212-c1805f53ffaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:21.119928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158914aa-2e6e-4b62-8212-c1805f53ffaf service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.120239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158914aa-2e6e-4b62-8212-c1805f53ffaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.120518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158914aa-2e6e-4b62-8212-c1805f53ffaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.137945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158914aa-2e6e-4b62-8212-c1805f53ffaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:21.139921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158914aa-2e6e-4b62-8212-c1805f53ffaf service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:21.157821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 976/1960] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:21.168060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc6b58cfc329475d9700fedf909884d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.175798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc6b58cfc329475d9700fedf909884d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.177043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc6b58cfc329475d9700fedf909884d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.186927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc6b58cfc329475d9700fedf909884d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.206442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc6b58cfc329475d9700fedf909884d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.215203 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 985/1961] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:20 2022] DELETE /identity/v3/users/fc6b58cfc329475d9700fedf909884d3 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:21.222158 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f3f09-c35e-4995-b048-874366ebee8f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.222691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f3f09-c35e-4995-b048-874366ebee8f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.223209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f3f09-c35e-4995-b048-874366ebee8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.247738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6b1f3f09-c35e-4995-b048-874366ebee8f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:21.250164 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6b1f3f09-c35e-4995-b048-874366ebee8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:21.392601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 977/1962] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:21.401030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:21.403034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:21.418682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236dd8-91bb-4b98-97c9-83c83d0082c9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:21.420926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f3f09-c35e-4995-b048-874366ebee8f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:21.422716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6b1f3f09-c35e-4995-b048-874366ebee8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:21.459025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:21.459910 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.460268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.460544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] PATH_INFO: `/v3/projects/518119da4a5c4df4a03c527344ed30cc` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.462913 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=518119da4a5c4df4a03c527344ed30cc)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:21.470021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:21.516176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:21.539331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:21.555211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f3f09-c35e-4995-b048-874366ebee8f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:21.562382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518119da4a5c4df4a03c527344ed30cc', 'request_id': 'req-881c822f-56df-4a32-94a7-3b9070edac36'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.571861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518119da4a5c4df4a03c527344ed30cc', 'request_id': 'req-881c822f-56df-4a32-94a7-3b9070edac36'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.583245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518119da4a5c4df4a03c527344ed30cc', 'request_id': 'req-881c822f-56df-4a32-94a7-3b9070edac36'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.596978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] The token cache is being invalidate because project 518119da4a5c4df4a03c527344ed30cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:21.597324 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.597979 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-881c822f-56df-4a32-94a7-3b9070edac36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.599555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 986/1963] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:21 2022] DELETE /identity/v3/projects/518119da4a5c4df4a03c527344ed30cc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:21.605490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98660d5-3561-4172-8b68-b180bab68471 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.605875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98660d5-3561-4172-8b68-b180bab68471 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.606227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98660d5-3561-4172-8b68-b180bab68471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.616081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0eeb6-e351-40d5-811f-c1c3fc1d28e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:21.617108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0eeb6-e351-40d5-811f-c1c3fc1d28e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.617108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0eeb6-e351-40d5-811f-c1c3fc1d28e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.617338 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0eeb6-e351-40d5-811f-c1c3fc1d28e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.619114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e2d0eeb6-e351-40d5-811f-c1c3fc1d28e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:21.625226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a98660d5-3561-4172-8b68-b180bab68471 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:21.626432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a98660d5-3561-4172-8b68-b180bab68471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:21.697754 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 987/1964] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:21.698626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98660d5-3561-4172-8b68-b180bab68471 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:21.699938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a98660d5-3561-4172-8b68-b180bab68471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:21.712318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98660d5-3561-4172-8b68-b180bab68471 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:21.726605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:21.727523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.727895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.728069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] PATH_INFO: `/v3/projects/feb2f85fc9d0484f827d5be0d58a3665` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.729298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feb2f85fc9d0484f827d5be0d58a3665)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:21.731241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:21.736934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0eeb6-e351-40d5-811f-c1c3fc1d28e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:21.738392 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0eeb6-e351-40d5-811f-c1c3fc1d28e2 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:21.752448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 978/1965] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:21.754456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c0fa-7efd-459d-be11-2acad8ba1dcb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.754782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c0fa-7efd-459d-be11-2acad8ba1dcb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.755057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c0fa-7efd-459d-be11-2acad8ba1dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.777228 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3103c0fa-7efd-459d-be11-2acad8ba1dcb None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:21.779429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3103c0fa-7efd-459d-be11-2acad8ba1dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:21.795432 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 979/1966] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:21.799562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103c0fa-7efd-459d-be11-2acad8ba1dcb None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:21.811646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103c0fa-7efd-459d-be11-2acad8ba1dcb None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:21.833462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feb2f85fc9d0484f827d5be0d58a3665', 'request_id': 'req-593c8363-6f0c-4bb2-b78d-727e34a6a211'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.845117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feb2f85fc9d0484f827d5be0d58a3665', 'request_id': 'req-593c8363-6f0c-4bb2-b78d-727e34a6a211'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.861084 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feb2f85fc9d0484f827d5be0d58a3665', 'request_id': 'req-593c8363-6f0c-4bb2-b78d-727e34a6a211'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.887408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] The token cache is being invalidate because project feb2f85fc9d0484f827d5be0d58a3665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:21.887799 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.889354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:21.891876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 988/1967] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:21 2022] DELETE /identity/v3/projects/feb2f85fc9d0484f827d5be0d58a3665 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:21.896237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:21.897810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:21.903518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdec1e-58f1-4bbb-8f9a-03fb3f4c6819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:21.904416 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdec1e-58f1-4bbb-8f9a-03fb3f4c6819 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.904569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdec1e-58f1-4bbb-8f9a-03fb3f4c6819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.906879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdec1e-58f1-4bbb-8f9a-03fb3f4c6819 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.907658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cdec1e-58f1-4bbb-8f9a-03fb3f4c6819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:21.907972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cdec1e-58f1-4bbb-8f9a-03fb3f4c6819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:21.909690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cdec1e-58f1-4bbb-8f9a-03fb3f4c6819 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:21.913827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c8363-6f0c-4bb2-b78d-727e34a6a211 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:21.921011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:21.922210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.922530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.922812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] PATH_INFO: `/v3/users/7e1efafdd5be4237beec9bcab2fb2045` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.924689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e1efafdd5be4237beec9bcab2fb2045)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:21.928820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:21.933586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 980/1968] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:21.938995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdec1e-58f1-4bbb-8f9a-03fb3f4c6819 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:21.944732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-76cdec1e-58f1-4bbb-8f9a-03fb3f4c6819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:21.973364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdec1e-58f1-4bbb-8f9a-03fb3f4c6819 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:21.982674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967901f3-cc1c-4bdc-9e79-8ef453bd2c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:21.983457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967901f3-cc1c-4bdc-9e79-8ef453bd2c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:21.983741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967901f3-cc1c-4bdc-9e79-8ef453bd2c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:21.984140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967901f3-cc1c-4bdc-9e79-8ef453bd2c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:21.984963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967901f3-cc1c-4bdc-9e79-8ef453bd2c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:21.987293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967901f3-cc1c-4bdc-9e79-8ef453bd2c14 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:22.005042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:22.028061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e1efafdd5be4237beec9bcab2fb2045'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.040274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e1efafdd5be4237beec9bcab2fb2045'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.052568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e1efafdd5be4237beec9bcab2fb2045'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.054996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 981/1969] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:21 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:22.059372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967901f3-cc1c-4bdc-9e79-8ef453bd2c14 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:22.075351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e1efafdd5be4237beec9bcab2fb2045'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.086634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:22.101747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e1efafdd5be4237beec9bcab2fb2045'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.112781 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 989/1970] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:21 2022] DELETE /identity/v3/users/7e1efafdd5be4237beec9bcab2fb2045 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:22.166079 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967901f3-cc1c-4bdc-9e79-8ef453bd2c14 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:22.248553 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb156e-7862-4796-a988-8224a325d38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:22.249428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb156e-7862-4796-a988-8224a325d38c admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:22.249774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb156e-7862-4796-a988-8224a325d38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:22.250663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb156e-7862-4796-a988-8224a325d38c admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:22.251935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb156e-7862-4796-a988-8224a325d38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:22.255585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb156e-7862-4796-a988-8224a325d38c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:22.298869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 982/1971] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:22 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:22.302559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:22.303997 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:22.308848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb156e-7862-4796-a988-8224a325d38c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:22.310788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b4cb156e-7862-4796-a988-8224a325d38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:22.318886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21f2a9-f1de-45ed-80f3-c040af6fcf06 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:22.324956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb156e-7862-4796-a988-8224a325d38c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:22.329856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:22.331291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:22.331291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:22.331500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] PATH_INFO: `/v3/users/bad8ee8b4344457b861429ca235c824f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:22.332574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd38ecd-6db9-4301-8c08-81b4f56bb594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:22.332699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad8ee8b4344457b861429ca235c824f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:22.334134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd38ecd-6db9-4301-8c08-81b4f56bb594 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:22.334667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd38ecd-6db9-4301-8c08-81b4f56bb594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:22.334924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:22.335727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd38ecd-6db9-4301-8c08-81b4f56bb594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:22.337380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd38ecd-6db9-4301-8c08-81b4f56bb594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:22.339682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd38ecd-6db9-4301-8c08-81b4f56bb594 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:22.353680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 983/1972] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:22.360696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd38ecd-6db9-4301-8c08-81b4f56bb594 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:22.374021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd38ecd-6db9-4301-8c08-81b4f56bb594 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:22.380936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf469c-64e3-4709-a9a4-4562f28cfc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:22.382046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf469c-64e3-4709-a9a4-4562f28cfc32 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:22.382825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf469c-64e3-4709-a9a4-4562f28cfc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:22.383278 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf469c-64e3-4709-a9a4-4562f28cfc32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:22.385138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf469c-64e3-4709-a9a4-4562f28cfc32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:22.387525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf469c-64e3-4709-a9a4-4562f28cfc32 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:22.404021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 984/1973] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:22.406482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf469c-64e3-4709-a9a4-4562f28cfc32 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:22.412477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad8ee8b4344457b861429ca235c824f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.421561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad8ee8b4344457b861429ca235c824f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.422338 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad8ee8b4344457b861429ca235c824f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.425269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf469c-64e3-4709-a9a4-4562f28cfc32 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:22.432580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7ebb5-6b57-4046-a00c-b636095d8004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:22.433841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7ebb5-6b57-4046-a00c-b636095d8004 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:22.434306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7ebb5-6b57-4046-a00c-b636095d8004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:22.434751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7ebb5-6b57-4046-a00c-b636095d8004 admin admin] PATH_INFO: `/v3/projects/229277f47d26424fab98e71d968cf35f/users/cfc149fdd1ac453a953039ea2f259cae/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:22.435229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad8ee8b4344457b861429ca235c824f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.453543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad8ee8b4344457b861429ca235c824f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.455861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7ebb5-6b57-4046-a00c-b636095d8004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=229277f47d26424fab98e71d968cf35f, user_id=cfc149fdd1ac453a953039ea2f259cae, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:22.459212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7ebb5-6b57-4046-a00c-b636095d8004 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:22.463306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 990/1974] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:22 2022] DELETE /identity/v3/users/bad8ee8b4344457b861429ca235c824f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:22.464796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:22.467231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:22.488721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 985/1975] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:22 2022] PUT /identity/v3/projects/229277f47d26424fab98e71d968cf35f/users/cfc149fdd1ac453a953039ea2f259cae/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:22.515705 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7760a09-6543-4ea0-8d4c-98ffa45e6784 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:22.516645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7760a09-6543-4ea0-8d4c-98ffa45e6784 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:22.518280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7760a09-6543-4ea0-8d4c-98ffa45e6784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:22.550901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c7760a09-6543-4ea0-8d4c-98ffa45e6784 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:22.552188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c7760a09-6543-4ea0-8d4c-98ffa45e6784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:22.607808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a66b01-e042-4367-a8ee-6d0eff6a32c0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:22.640912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:22.669748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 986/1976] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:22.671833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c418411-945a-41c8-896b-c4a4b429deec None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:22.672430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c418411-945a-41c8-896b-c4a4b429deec None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:22.672853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c418411-945a-41c8-896b-c4a4b429deec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:22.679889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:22.680827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:22.681110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:22.681384 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] PATH_INFO: `/v3/projects/61a67646f4ef4d768d656bc1be25adc2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:22.682970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a67646f4ef4d768d656bc1be25adc2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:22.684135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:22.684953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:22.718772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2c418411-945a-41c8-896b-c4a4b429deec None None] MFA Rules not processed for user `cfc149fdd1ac453a953039ea2f259cae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:22.720538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2c418411-945a-41c8-896b-c4a4b429deec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:22.754759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a67646f4ef4d768d656bc1be25adc2', 'request_id': 'req-3913db00-925d-4cfb-a36e-9e7626277141'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.763571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a67646f4ef4d768d656bc1be25adc2', 'request_id': 'req-3913db00-925d-4cfb-a36e-9e7626277141'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.792269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a67646f4ef4d768d656bc1be25adc2', 'request_id': 'req-3913db00-925d-4cfb-a36e-9e7626277141'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.816328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] The token cache is being invalidate because project 61a67646f4ef4d768d656bc1be25adc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:22.819239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.819876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:22.824008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 991/1977] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:22 2022] DELETE /identity/v3/projects/61a67646f4ef4d768d656bc1be25adc2 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:22.832205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:22.838318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:22.871296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 987/1978] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:22.875632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c418411-945a-41c8-896b-c4a4b429deec None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:22.879477 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2c418411-945a-41c8-896b-c4a4b429deec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:22.909389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c418411-945a-41c8-896b-c4a4b429deec None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:22.943475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913db00-925d-4cfb-a36e-9e7626277141 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:22.948306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6105990-9966-4881-974f-9ef2a23c2b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:22.949643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6105990-9966-4881-974f-9ef2a23c2b16 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:22.949918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6105990-9966-4881-974f-9ef2a23c2b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:22.950201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6105990-9966-4881-974f-9ef2a23c2b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:22.953412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c6105990-9966-4881-974f-9ef2a23c2b16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:23.083781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:23.085941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:23.086542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:23.087133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] PATH_INFO: `/v3/users/c722117ea9504830b31e25dd3734d7a8` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:23.091068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c722117ea9504830b31e25dd3734d7a8)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:23.094311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:23.106659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6105990-9966-4881-974f-9ef2a23c2b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:23.109549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6105990-9966-4881-974f-9ef2a23c2b16 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:23.131148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 992/1979] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:23.131985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6105990-9966-4881-974f-9ef2a23c2b16 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:23.133468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c6105990-9966-4881-974f-9ef2a23c2b16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:23.145716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6105990-9966-4881-974f-9ef2a23c2b16 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:23.152678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7d074-f4b1-4a19-9fba-5441834aa92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:23.153588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7d074-f4b1-4a19-9fba-5441834aa92c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:23.153885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7d074-f4b1-4a19-9fba-5441834aa92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:23.154309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7d074-f4b1-4a19-9fba-5441834aa92c admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:23.154840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7d074-f4b1-4a19-9fba-5441834aa92c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:23.157525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7d074-f4b1-4a19-9fba-5441834aa92c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:23.161673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:23.167172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c722117ea9504830b31e25dd3734d7a8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.168248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c722117ea9504830b31e25dd3734d7a8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.178865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c722117ea9504830b31e25dd3734d7a8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.180532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c722117ea9504830b31e25dd3734d7a8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.195724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c722117ea9504830b31e25dd3734d7a8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.201745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 993/1980] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:23 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:23.202836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7d074-f4b1-4a19-9fba-5441834aa92c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:23.204845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-02a7d074-f4b1-4a19-9fba-5441834aa92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:23.215469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 988/1981] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:22 2022] DELETE /identity/v3/users/c722117ea9504830b31e25dd3734d7a8 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:23.218998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:23.233366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:23.280676 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:23.370407 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7d074-f4b1-4a19-9fba-5441834aa92c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:23.412456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a8261-76df-4780-ab14-d4b2c894608e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:23.431251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e4043-04d7-45be-bf2e-f22f2c12ad37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:23.432373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e4043-04d7-45be-bf2e-f22f2c12ad37 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:23.432655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e4043-04d7-45be-bf2e-f22f2c12ad37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:23.432929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e4043-04d7-45be-bf2e-f22f2c12ad37 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:23.433644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e4043-04d7-45be-bf2e-f22f2c12ad37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:23.439897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e4043-04d7-45be-bf2e-f22f2c12ad37 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:23.490320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 989/1982] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:23 2022] POST /identity/v3/users => generated 299 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:23.491366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:23.492316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:23.492603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:23.493271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] PATH_INFO: `/v3/projects/170af42ec61f4dbbbf6ba9606848001d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:23.494901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=170af42ec61f4dbbbf6ba9606848001d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:23.495431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e4043-04d7-45be-bf2e-f22f2c12ad37 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:23.497566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:23.537213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e4043-04d7-45be-bf2e-f22f2c12ad37 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:23.554535 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da018e45-8b36-4a79-9550-ea2221ddbafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:23.555259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da018e45-8b36-4a79-9550-ea2221ddbafd admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:23.555547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da018e45-8b36-4a79-9550-ea2221ddbafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:23.555856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da018e45-8b36-4a79-9550-ea2221ddbafd admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:23.556992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da018e45-8b36-4a79-9550-ea2221ddbafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:23.558630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da018e45-8b36-4a79-9550-ea2221ddbafd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:23.576327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170af42ec61f4dbbbf6ba9606848001d', 'request_id': 'req-8e281291-4a9c-4cad-930d-ca529c3fb38f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.577103 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 990/1983] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:23.581745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da018e45-8b36-4a79-9550-ea2221ddbafd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:23.590301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170af42ec61f4dbbbf6ba9606848001d', 'request_id': 'req-8e281291-4a9c-4cad-930d-ca529c3fb38f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.593168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da018e45-8b36-4a79-9550-ea2221ddbafd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:23.603035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170af42ec61f4dbbbf6ba9606848001d', 'request_id': 'req-8e281291-4a9c-4cad-930d-ca529c3fb38f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.642616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] The token cache is being invalidate because project 170af42ec61f4dbbbf6ba9606848001d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:23.643100 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.645244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8e281291-4a9c-4cad-930d-ca529c3fb38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.648452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 994/1984] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:23 2022] DELETE /identity/v3/projects/170af42ec61f4dbbbf6ba9606848001d => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:23.657696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c456b0-7f5b-46b9-b77e-a4f7f1f5a5a1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:23.657987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c456b0-7f5b-46b9-b77e-a4f7f1f5a5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:23.658268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c456b0-7f5b-46b9-b77e-a4f7f1f5a5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:23.685436 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5a945-b59e-4822-b657-afcc3e659549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:23.693606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a945-b59e-4822-b657-afcc3e659549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:23.693932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a945-b59e-4822-b657-afcc3e659549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:23.694215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a945-b59e-4822-b657-afcc3e659549 admin admin] PATH_INFO: `/v3/projects/7058801184ba4f94a7b6ee0cccadccb4/users/bbf0fd1e96904f939d50996e1143b31d/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:23.701983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e5c456b0-7f5b-46b9-b77e-a4f7f1f5a5a1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:23.706578 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e5c456b0-7f5b-46b9-b77e-a4f7f1f5a5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:23.724283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5a945-b59e-4822-b657-afcc3e659549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7058801184ba4f94a7b6ee0cccadccb4, user_id=bbf0fd1e96904f939d50996e1143b31d, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:23.727063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5a945-b59e-4822-b657-afcc3e659549 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:23.729019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 995/1985] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:23.738198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c456b0-7f5b-46b9-b77e-a4f7f1f5a5a1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:23.739766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e5c456b0-7f5b-46b9-b77e-a4f7f1f5a5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:23.754021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c456b0-7f5b-46b9-b77e-a4f7f1f5a5a1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:23.770337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:23.779249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 991/1986] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:23 2022] PUT /identity/v3/projects/7058801184ba4f94a7b6ee0cccadccb4/users/bbf0fd1e96904f939d50996e1143b31d/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:23.780670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2266a12-8e6b-43c0-b63c-06d67b0030c2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:23.780959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2266a12-8e6b-43c0-b63c-06d67b0030c2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:23.781232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2266a12-8e6b-43c0-b63c-06d67b0030c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:23.805976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b2266a12-8e6b-43c0-b63c-06d67b0030c2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:23.807648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b2266a12-8e6b-43c0-b63c-06d67b0030c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:23.842432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:23.843366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:23.844192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:23.844470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] PATH_INFO: `/v3/projects/7dbd5e8bc3f74241ae033e620f2f9c90` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:23.846758 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dbd5e8bc3f74241ae033e620f2f9c90)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:23.850149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:23.892281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 992/1987] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:23.893241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2266a12-8e6b-43c0-b63c-06d67b0030c2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:23.895059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b2266a12-8e6b-43c0-b63c-06d67b0030c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:23.906998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2266a12-8e6b-43c0-b63c-06d67b0030c2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:23.907865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dbd5e8bc3f74241ae033e620f2f9c90', 'request_id': 'req-9a5b642c-7912-400f-ae5e-6bfaa936423f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.915526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dbd5e8bc3f74241ae033e620f2f9c90', 'request_id': 'req-9a5b642c-7912-400f-ae5e-6bfaa936423f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.926856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:23.929307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dbd5e8bc3f74241ae033e620f2f9c90', 'request_id': 'req-9a5b642c-7912-400f-ae5e-6bfaa936423f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.944134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] The token cache is being invalidate because project 7dbd5e8bc3f74241ae033e620f2f9c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:23.944502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.945256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:23.947419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 996/1988] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:23 2022] DELETE /identity/v3/projects/7dbd5e8bc3f74241ae033e620f2f9c90 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:23.948212 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:23.957454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:23.983536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff35017-0386-4a59-a75b-c89e43acbb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:23.984261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff35017-0386-4a59-a75b-c89e43acbb54 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:23.984554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff35017-0386-4a59-a75b-c89e43acbb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:23.984823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff35017-0386-4a59-a75b-c89e43acbb54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:23.985563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff35017-0386-4a59-a75b-c89e43acbb54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:23.987855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff35017-0386-4a59-a75b-c89e43acbb54 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:23.998236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 993/1989] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:23.998980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff35017-0386-4a59-a75b-c89e43acbb54 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.000706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-eff35017-0386-4a59-a75b-c89e43acbb54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:24.014355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff35017-0386-4a59-a75b-c89e43acbb54 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.020527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.022134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.022772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.023188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] PATH_INFO: `/v3/projects/abb1c4c127864f8db7bb2be0880141d7` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.025545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb1c4c127864f8db7bb2be0880141d7)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.029062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.033412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b642c-7912-400f-ae5e-6bfaa936423f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.038870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03a196-b3e4-4abe-aed5-aacb1d1a5d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.039553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a196-b3e4-4abe-aed5-aacb1d1a5d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.039840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a196-b3e4-4abe-aed5-aacb1d1a5d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.040134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a196-b3e4-4abe-aed5-aacb1d1a5d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.041484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03a196-b3e4-4abe-aed5-aacb1d1a5d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:24.041800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03a196-b3e4-4abe-aed5-aacb1d1a5d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.043440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03a196-b3e4-4abe-aed5-aacb1d1a5d50 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.058921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 997/1990] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:24.063026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03a196-b3e4-4abe-aed5-aacb1d1a5d50 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.064337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa03a196-b3e4-4abe-aed5-aacb1d1a5d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:24.079295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03a196-b3e4-4abe-aed5-aacb1d1a5d50 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.086247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.087211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.087501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.087783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] PATH_INFO: `/v3/users/f4ca8304f1c94a309e087b4dab970f86` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.089080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ca8304f1c94a309e087b4dab970f86)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.095294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.100943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb1c4c127864f8db7bb2be0880141d7', 'request_id': 'req-edb13ad7-fc9b-4e91-b85d-20656e8c715c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.112402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb1c4c127864f8db7bb2be0880141d7', 'request_id': 'req-edb13ad7-fc9b-4e91-b85d-20656e8c715c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.124587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb1c4c127864f8db7bb2be0880141d7', 'request_id': 'req-edb13ad7-fc9b-4e91-b85d-20656e8c715c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.136962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] The token cache is being invalidate because project abb1c4c127864f8db7bb2be0880141d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:24.137250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.139237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.140897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 994/1991] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:23 2022] DELETE /identity/v3/projects/abb1c4c127864f8db7bb2be0880141d7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:24.145248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.145248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:24.159172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:24.176673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ca8304f1c94a309e087b4dab970f86'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.188860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ca8304f1c94a309e087b4dab970f86'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.190426 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ca8304f1c94a309e087b4dab970f86'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.203509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ca8304f1c94a309e087b4dab970f86'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.239405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ca8304f1c94a309e087b4dab970f86'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.248498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 998/1992] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:24 2022] DELETE /identity/v3/users/f4ca8304f1c94a309e087b4dab970f86 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:24.249756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.252981 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:24.256556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb13ad7-fc9b-4e91-b85d-20656e8c715c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.317401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:24.337793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb2184-ec6e-4b7f-b251-b5246c4662fd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.344235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7df96f-db51-40b6-a91c-ca61b7f9e826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.344988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7df96f-db51-40b6-a91c-ca61b7f9e826 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.345280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7df96f-db51-40b6-a91c-ca61b7f9e826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.345554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7df96f-db51-40b6-a91c-ca61b7f9e826 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.346764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7df96f-db51-40b6-a91c-ca61b7f9e826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.349727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7df96f-db51-40b6-a91c-ca61b7f9e826 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.350591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170684a-7ff1-4fd6-8b7b-4f50fe92bb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.351916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170684a-7ff1-4fd6-8b7b-4f50fe92bb22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.352362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170684a-7ff1-4fd6-8b7b-4f50fe92bb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.352772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170684a-7ff1-4fd6-8b7b-4f50fe92bb22 admin admin] PATH_INFO: `/v3/projects/7058801184ba4f94a7b6ee0cccadccb4/users/bbf0fd1e96904f939d50996e1143b31d/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.357510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170684a-7ff1-4fd6-8b7b-4f50fe92bb22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7058801184ba4f94a7b6ee0cccadccb4, user_id=bbf0fd1e96904f939d50996e1143b31d, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.362111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170684a-7ff1-4fd6-8b7b-4f50fe92bb22 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.376077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 995/1993] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:24 2022] PUT /identity/v3/projects/7058801184ba4f94a7b6ee0cccadccb4/users/bbf0fd1e96904f939d50996e1143b31d/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:24.377950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abc750-51c4-4610-8e55-9dd6c5bad070 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.378790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abc750-51c4-4610-8e55-9dd6c5bad070 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.379066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abc750-51c4-4610-8e55-9dd6c5bad070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.383194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 999/1994] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:24 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:24.386183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf26db-de54-4b30-a04a-00f40323ef7a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.386754 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf26db-de54-4b30-a04a-00f40323ef7a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.387032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf26db-de54-4b30-a04a-00f40323ef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.404660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c9abc750-51c4-4610-8e55-9dd6c5bad070 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:24.405871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c9abc750-51c4-4610-8e55-9dd6c5bad070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:24.415195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5fdf26db-de54-4b30-a04a-00f40323ef7a None None] MFA Rules not processed for user `bbf0fd1e96904f939d50996e1143b31d`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:24.416547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5fdf26db-de54-4b30-a04a-00f40323ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:24.476603 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 996/1995] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:24.477698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abc750-51c4-4610-8e55-9dd6c5bad070 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.491651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abc750-51c4-4610-8e55-9dd6c5bad070 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.498082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1000/1996] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:24.498082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc032111-a571-49c3-bcda-e665b9b66d2d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.498082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc032111-a571-49c3-bcda-e665b9b66d2d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.498082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc032111-a571-49c3-bcda-e665b9b66d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.501764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee3558-7b34-46de-b750-be104f5bc1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.502580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3558-7b34-46de-b750-be104f5bc1f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.502931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3558-7b34-46de-b750-be104f5bc1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.503302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3558-7b34-46de-b750-be104f5bc1f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.504174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee3558-7b34-46de-b750-be104f5bc1f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.506500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee3558-7b34-46de-b750-be104f5bc1f1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.518641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bc032111-a571-49c3-bcda-e665b9b66d2d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:24.519953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bc032111-a571-49c3-bcda-e665b9b66d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:24.528472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1001/1997] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:24.529344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc032111-a571-49c3-bcda-e665b9b66d2d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.532688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 997/1998] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:24 2022] POST /identity/v3/users => generated 293 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:24.532953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bc032111-a571-49c3-bcda-e665b9b66d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:24.534457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee3558-7b34-46de-b750-be104f5bc1f1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.546796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee3558-7b34-46de-b750-be104f5bc1f1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.551942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:24.553344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034de7b-388d-4c70-a4aa-3904113bd1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.554304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034de7b-388d-4c70-a4aa-3904113bd1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.555096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034de7b-388d-4c70-a4aa-3904113bd1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.555454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034de7b-388d-4c70-a4aa-3904113bd1e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.556984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034de7b-388d-4c70-a4aa-3904113bd1e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:24.557496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034de7b-388d-4c70-a4aa-3904113bd1e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.559194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034de7b-388d-4c70-a4aa-3904113bd1e3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.574770 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 998/1999] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:24.577110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034de7b-388d-4c70-a4aa-3904113bd1e3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.589963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034de7b-388d-4c70-a4aa-3904113bd1e3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.594925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df443-4191-4061-ad07-7d2af8078e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.595718 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df443-4191-4061-ad07-7d2af8078e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.596100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df443-4191-4061-ad07-7d2af8078e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.596513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df443-4191-4061-ad07-7d2af8078e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.597444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830df443-4191-4061-ad07-7d2af8078e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.599167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830df443-4191-4061-ad07-7d2af8078e92 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.601010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc032111-a571-49c3-bcda-e665b9b66d2d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.606492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f451e0-7786-401a-8430-0418c65505ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.607204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f451e0-7786-401a-8430-0418c65505ce service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.607630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f451e0-7786-401a-8430-0418c65505ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.608014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f451e0-7786-401a-8430-0418c65505ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.614065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 999/2000] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:24.614065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df443-4191-4061-ad07-7d2af8078e92 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.622319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f451e0-7786-401a-8430-0418c65505ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.623991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f451e0-7786-401a-8430-0418c65505ce service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.624135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df443-4191-4061-ad07-7d2af8078e92 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.630065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393efdac-30ef-4610-8b9d-f6b9b546a191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.630884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393efdac-30ef-4610-8b9d-f6b9b546a191 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.631066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393efdac-30ef-4610-8b9d-f6b9b546a191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.631339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393efdac-30ef-4610-8b9d-f6b9b546a191 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.632023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393efdac-30ef-4610-8b9d-f6b9b546a191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.634350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393efdac-30ef-4610-8b9d-f6b9b546a191 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.636513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1002/2001] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:24.637328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f451e0-7786-401a-8430-0418c65505ce service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.657388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f451e0-7786-401a-8430-0418c65505ce service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.662585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7491d-1da9-411f-af98-504db5ff2c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.663272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7491d-1da9-411f-af98-504db5ff2c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.663552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7491d-1da9-411f-af98-504db5ff2c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.663820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7491d-1da9-411f-af98-504db5ff2c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.664610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7491d-1da9-411f-af98-504db5ff2c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.666411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7491d-1da9-411f-af98-504db5ff2c30 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.682764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1000/2002] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:24 2022] POST /identity/v3/projects => generated 359 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:24.686844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393efdac-30ef-4610-8b9d-f6b9b546a191 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.709676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1003/2003] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:24.711756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7491d-1da9-411f-af98-504db5ff2c30 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.714054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e3c7491d-1da9-411f-af98-504db5ff2c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:24.728051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7491d-1da9-411f-af98-504db5ff2c30 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.795063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393efdac-30ef-4610-8b9d-f6b9b546a191 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.801891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4ab31-a23e-4ca9-a814-b5f7f11f8b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.802583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4ab31-a23e-4ca9-a814-b5f7f11f8b6d admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.802866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4ab31-a23e-4ca9-a814-b5f7f11f8b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.803149 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4ab31-a23e-4ca9-a814-b5f7f11f8b6d admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.803868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4ab31-a23e-4ca9-a814-b5f7f11f8b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.805829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4ab31-a23e-4ca9-a814-b5f7f11f8b6d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.813479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.814392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.814669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.815038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] PATH_INFO: `/v3/users/cfc149fdd1ac453a953039ea2f259cae` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.816384 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc149fdd1ac453a953039ea2f259cae)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.818419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.849664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1001/2004] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:24 2022] POST /identity/v3/users => generated 280 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:24.850590 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4ab31-a23e-4ca9-a814-b5f7f11f8b6d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.856220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:24.862466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4ab31-a23e-4ca9-a814-b5f7f11f8b6d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:24.871297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a44f0-9de4-4554-92cf-d93c58f26af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:24.872312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a44f0-9de4-4554-92cf-d93c58f26af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.872604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a44f0-9de4-4554-92cf-d93c58f26af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.872884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a44f0-9de4-4554-92cf-d93c58f26af6 admin admin] PATH_INFO: `/v3/projects/a2335bd0982042e4a907a2fdcc60f869/users/85bdda041ef243f8b77204be2924482f/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.883623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc149fdd1ac453a953039ea2f259cae'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.890896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a44f0-9de4-4554-92cf-d93c58f26af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2335bd0982042e4a907a2fdcc60f869, user_id=85bdda041ef243f8b77204be2924482f, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:24.891036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc149fdd1ac453a953039ea2f259cae'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.891662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc149fdd1ac453a953039ea2f259cae'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.893904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a44f0-9de4-4554-92cf-d93c58f26af6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:24.901647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc149fdd1ac453a953039ea2f259cae'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.910202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1002/2005] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:24 2022] PUT /identity/v3/projects/a2335bd0982042e4a907a2fdcc60f869/users/85bdda041ef243f8b77204be2924482f/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:24.910821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a44f0-9de4-4554-92cf-d93c58f26af6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:24.920437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5c59bae5-db1e-4830-ae54-a0da514b690c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc149fdd1ac453a953039ea2f259cae'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:24.929019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1004/2006] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:24 2022] DELETE /identity/v3/users/cfc149fdd1ac453a953039ea2f259cae => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:24.930955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b1b16-19ee-429e-94ee-247e41655423 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:24.931824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b1b16-19ee-429e-94ee-247e41655423 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:24.932832 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b1b16-19ee-429e-94ee-247e41655423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:24.963476 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1c5b1b16-19ee-429e-94ee-247e41655423 None None] MFA Rules not processed for user `85bdda041ef243f8b77204be2924482f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:24.965099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1c5b1b16-19ee-429e-94ee-247e41655423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:24.980532 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:25.024856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a44f0-9de4-4554-92cf-d93c58f26af6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.041640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1005/2007] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:25.043143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692978aa-ecc3-4794-afdf-7d4438c228d9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.043481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692978aa-ecc3-4794-afdf-7d4438c228d9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.043764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692978aa-ecc3-4794-afdf-7d4438c228d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.062667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-692978aa-ecc3-4794-afdf-7d4438c228d9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:25.063889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-692978aa-ecc3-4794-afdf-7d4438c228d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.085512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95a908-c780-4a3d-ab9e-cf19b53c9835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.086188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a908-c780-4a3d-ab9e-cf19b53c9835 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.086466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a908-c780-4a3d-ab9e-cf19b53c9835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.086736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a908-c780-4a3d-ab9e-cf19b53c9835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.087471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95a908-c780-4a3d-ab9e-cf19b53c9835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.089108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95a908-c780-4a3d-ab9e-cf19b53c9835 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.099422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1003/2008] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:25.099938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95a908-c780-4a3d-ab9e-cf19b53c9835 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.101566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1e95a908-c780-4a3d-ab9e-cf19b53c9835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.113778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95a908-c780-4a3d-ab9e-cf19b53c9835 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.118500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.119351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.119629 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.119898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] PATH_INFO: `/v3/projects/cab69e10991246f48b060241ea7006f0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.121182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab69e10991246f48b060241ea7006f0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.123083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.124823 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1006/2009] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:25.129041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692978aa-ecc3-4794-afdf-7d4438c228d9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.130759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-692978aa-ecc3-4794-afdf-7d4438c228d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.142901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692978aa-ecc3-4794-afdf-7d4438c228d9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.147839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff9e3a-7c06-47b5-b57d-3f9236225802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.148542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff9e3a-7c06-47b5-b57d-3f9236225802 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.148821 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff9e3a-7c06-47b5-b57d-3f9236225802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.149240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff9e3a-7c06-47b5-b57d-3f9236225802 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.149904 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff9e3a-7c06-47b5-b57d-3f9236225802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.152068 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff9e3a-7c06-47b5-b57d-3f9236225802 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.171581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab69e10991246f48b060241ea7006f0', 'request_id': 'req-35d01222-fa96-4fe3-aa8a-f667a909326a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:25.182485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1007/2010] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:25 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:25.182644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab69e10991246f48b060241ea7006f0', 'request_id': 'req-35d01222-fa96-4fe3-aa8a-f667a909326a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:25.182756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff9e3a-7c06-47b5-b57d-3f9236225802 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.182756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-89ff9e3a-7c06-47b5-b57d-3f9236225802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.189675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab69e10991246f48b060241ea7006f0', 'request_id': 'req-35d01222-fa96-4fe3-aa8a-f667a909326a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:25.200325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] The token cache is being invalidate because project cab69e10991246f48b060241ea7006f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:25.200606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:25.201240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:25.202824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1004/2011] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:25 2022] DELETE /identity/v3/projects/cab69e10991246f48b060241ea7006f0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:25.203572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.205063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.211484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:25.247232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff9e3a-7c06-47b5-b57d-3f9236225802 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.252065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56f05d-a2f1-4e0f-8222-0640f5b1ab56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.252902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56f05d-a2f1-4e0f-8222-0640f5b1ab56 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.253124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56f05d-a2f1-4e0f-8222-0640f5b1ab56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.253401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56f05d-a2f1-4e0f-8222-0640f5b1ab56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.254136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56f05d-a2f1-4e0f-8222-0640f5b1ab56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.255804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56f05d-a2f1-4e0f-8222-0640f5b1ab56 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.266162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d01222-fa96-4fe3-aa8a-f667a909326a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.268696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1008/2012] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:25.269462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56f05d-a2f1-4e0f-8222-0640f5b1ab56 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.273307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4b56f05d-a2f1-4e0f-8222-0640f5b1ab56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.286783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56f05d-a2f1-4e0f-8222-0640f5b1ab56 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.292382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb968d1-cb54-4cd4-ad84-2293d95dee31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.293118 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb968d1-cb54-4cd4-ad84-2293d95dee31 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.293338 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb968d1-cb54-4cd4-ad84-2293d95dee31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.293612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb968d1-cb54-4cd4-ad84-2293d95dee31 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.294309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb968d1-cb54-4cd4-ad84-2293d95dee31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.296457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb968d1-cb54-4cd4-ad84-2293d95dee31 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.325808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1009/2013] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:25 2022] POST /identity/v3/users => generated 293 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:25.327213 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5280c-35d6-492d-9d1b-ceafeb154e3c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.327497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5280c-35d6-492d-9d1b-ceafeb154e3c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.327852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5280c-35d6-492d-9d1b-ceafeb154e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.348558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b4c5280c-35d6-492d-9d1b-ceafeb154e3c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:25.349507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c1de5-246d-4f3d-8425-3e13ed9a5cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.349785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b4c5280c-35d6-492d-9d1b-ceafeb154e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.350183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c1de5-246d-4f3d-8425-3e13ed9a5cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.350462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c1de5-246d-4f3d-8425-3e13ed9a5cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.350733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c1de5-246d-4f3d-8425-3e13ed9a5cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.352736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cd7c1de5-246d-4f3d-8425-3e13ed9a5cc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.364914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1010/2014] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:25.364914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5280c-35d6-492d-9d1b-ceafeb154e3c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.364914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b4c5280c-35d6-492d-9d1b-ceafeb154e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.371944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c1de5-246d-4f3d-8425-3e13ed9a5cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.373328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5280c-35d6-492d-9d1b-ceafeb154e3c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.373582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c1de5-246d-4f3d-8425-3e13ed9a5cc3 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.378185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab687e1c-a41d-4980-ba5c-dd1069b3c6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.379367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab687e1c-a41d-4980-ba5c-dd1069b3c6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.379367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab687e1c-a41d-4980-ba5c-dd1069b3c6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.379662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab687e1c-a41d-4980-ba5c-dd1069b3c6dc admin admin] PATH_INFO: `/v3/projects/a250614940da4b42aaf0477dd64ff0f3/users/250fc7108f0d44fb8e98c3c16e0296a7/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.382189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:25.387397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1005/2015] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:25.388073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c1de5-246d-4f3d-8425-3e13ed9a5cc3 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.400506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c1de5-246d-4f3d-8425-3e13ed9a5cc3 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.401710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab687e1c-a41d-4980-ba5c-dd1069b3c6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a250614940da4b42aaf0477dd64ff0f3, user_id=250fc7108f0d44fb8e98c3c16e0296a7, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.404528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab687e1c-a41d-4980-ba5c-dd1069b3c6dc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.405843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48949da8-2ea7-4ef4-9e57-00cb236354f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.406728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949da8-2ea7-4ef4-9e57-00cb236354f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.407237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949da8-2ea7-4ef4-9e57-00cb236354f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.407646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949da8-2ea7-4ef4-9e57-00cb236354f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.408674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48949da8-2ea7-4ef4-9e57-00cb236354f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.410930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48949da8-2ea7-4ef4-9e57-00cb236354f2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.428192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1011/2016] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:25 2022] PUT /identity/v3/projects/a250614940da4b42aaf0477dd64ff0f3/users/250fc7108f0d44fb8e98c3c16e0296a7/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:25.428192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4aa6-7f6f-4deb-acc5-5d4931d334e0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.428192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4aa6-7f6f-4deb-acc5-5d4931d334e0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.428192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4aa6-7f6f-4deb-acc5-5d4931d334e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.453944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ee2e4aa6-7f6f-4deb-acc5-5d4931d334e0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:25.454937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1006/2017] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:25.456549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ee2e4aa6-7f6f-4deb-acc5-5d4931d334e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.458731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cfcaa-5671-4054-8544-daf47fadfeb4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.458731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cfcaa-5671-4054-8544-daf47fadfeb4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.458731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cfcaa-5671-4054-8544-daf47fadfeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.516631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4c7cfcaa-5671-4054-8544-daf47fadfeb4 None None] MFA Rules not processed for user `250fc7108f0d44fb8e98c3c16e0296a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:25.521403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4c7cfcaa-5671-4054-8544-daf47fadfeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.559652 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1012/2018] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:25.561262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e4aa6-7f6f-4deb-acc5-5d4931d334e0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.588892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e4aa6-7f6f-4deb-acc5-5d4931d334e0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.594838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c5b7d-66fd-47b0-a66c-7203d0ee34b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.596926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c5b7d-66fd-47b0-a66c-7203d0ee34b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.598046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c5b7d-66fd-47b0-a66c-7203d0ee34b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.598592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c5b7d-66fd-47b0-a66c-7203d0ee34b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.600225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c5b7d-66fd-47b0-a66c-7203d0ee34b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.603163 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c5b7d-66fd-47b0-a66c-7203d0ee34b3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.619707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1013/2019] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:25.621241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c5b7d-66fd-47b0-a66c-7203d0ee34b3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.649549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c5b7d-66fd-47b0-a66c-7203d0ee34b3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.661633 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92717f-3b43-4d60-bd72-7e4fa743f083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.662358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92717f-3b43-4d60-bd72-7e4fa743f083 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.662599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92717f-3b43-4d60-bd72-7e4fa743f083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.662873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92717f-3b43-4d60-bd72-7e4fa743f083 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.663619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92717f-3b43-4d60-bd72-7e4fa743f083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:25.664069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92717f-3b43-4d60-bd72-7e4fa743f083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.666114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92717f-3b43-4d60-bd72-7e4fa743f083 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.691894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1014/2020] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:25.692704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92717f-3b43-4d60-bd72-7e4fa743f083 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.704735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1007/2021] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:25.704735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7cfcaa-5671-4054-8544-daf47fadfeb4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.718054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7cfcaa-5671-4054-8544-daf47fadfeb4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.728666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92717f-3b43-4d60-bd72-7e4fa743f083 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.738015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16ac4b-1d37-49e8-9ca2-a4f059c5308b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.739360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16ac4b-1d37-49e8-9ca2-a4f059c5308b admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.739619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd582c-b062-4034-9ee7-5f7af0dc2411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.739774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16ac4b-1d37-49e8-9ca2-a4f059c5308b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.740145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16ac4b-1d37-49e8-9ca2-a4f059c5308b admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.740573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd582c-b062-4034-9ee7-5f7af0dc2411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.740973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16ac4b-1d37-49e8-9ca2-a4f059c5308b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.741069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd582c-b062-4034-9ee7-5f7af0dc2411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.741244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd582c-b062-4034-9ee7-5f7af0dc2411 admin admin] PATH_INFO: `/v3/projects/68b21a03c91b41ba8295de67c7606957/users/a42268e606ea44c7b3219da841f517a0/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.743078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16ac4b-1d37-49e8-9ca2-a4f059c5308b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.771024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd582c-b062-4034-9ee7-5f7af0dc2411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b21a03c91b41ba8295de67c7606957, user_id=a42268e606ea44c7b3219da841f517a0, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.774161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd582c-b062-4034-9ee7-5f7af0dc2411 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.777885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1008/2022] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:25 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:25.779367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16ac4b-1d37-49e8-9ca2-a4f059c5308b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.782770 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ce16ac4b-1d37-49e8-9ca2-a4f059c5308b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:25.794717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1015/2023] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:25 2022] PUT /identity/v3/projects/68b21a03c91b41ba8295de67c7606957/users/a42268e606ea44c7b3219da841f517a0/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:25.795432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd582c-b062-4034-9ee7-5f7af0dc2411 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:25.830066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:25.901985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16ac4b-1d37-49e8-9ca2-a4f059c5308b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.931348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd582c-b062-4034-9ee7-5f7af0dc2411 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:25.942315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a27411-a8e6-4800-b21d-1321501e32c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:25.943142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a27411-a8e6-4800-b21d-1321501e32c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.943710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a27411-a8e6-4800-b21d-1321501e32c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.944139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a27411-a8e6-4800-b21d-1321501e32c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:25.945616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a27411-a8e6-4800-b21d-1321501e32c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:25.950709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a27411-a8e6-4800-b21d-1321501e32c6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:25.985757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1016/2024] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:25 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:25.987326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0966b-30b6-46d0-b129-e05fb3ee8cd9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:25.988048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0966b-30b6-46d0-b129-e05fb3ee8cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:25.988427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0966b-30b6-46d0-b129-e05fb3ee8cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.014729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26af2b6-58fa-439a-9cc4-d19763f10134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.017810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-70c0966b-30b6-46d0-b129-e05fb3ee8cd9 None None] MFA Rules not processed for user `a42268e606ea44c7b3219da841f517a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:26.019621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-70c0966b-30b6-46d0-b129-e05fb3ee8cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.020250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26af2b6-58fa-439a-9cc4-d19763f10134 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.020689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26af2b6-58fa-439a-9cc4-d19763f10134 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.021102 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26af2b6-58fa-439a-9cc4-d19763f10134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.047856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:26.095439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26af2b6-58fa-439a-9cc4-d19763f10134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.097215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26af2b6-58fa-439a-9cc4-d19763f10134 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.106823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1009/2025] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:25 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:26.107726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26af2b6-58fa-439a-9cc4-d19763f10134 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.109675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e26af2b6-58fa-439a-9cc4-d19763f10134 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.116226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1017/2026] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:26.117322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0966b-30b6-46d0-b129-e05fb3ee8cd9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.125413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26af2b6-58fa-439a-9cc4-d19763f10134 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.129308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0966b-30b6-46d0-b129-e05fb3ee8cd9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.130134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.131001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.131406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.131683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] PATH_INFO: `/v3/projects/229277f47d26424fab98e71d968cf35f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.132879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=229277f47d26424fab98e71d968cf35f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.134781 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7fc1-0834-4e69-bd70-01c6d15b2a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.134920 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.135743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7fc1-0834-4e69-bd70-01c6d15b2a52 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.136186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7fc1-0834-4e69-bd70-01c6d15b2a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.136790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7fc1-0834-4e69-bd70-01c6d15b2a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.137887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7fc1-0834-4e69-bd70-01c6d15b2a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.139517 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7fc1-0834-4e69-bd70-01c6d15b2a52 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.151004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1018/2027] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:26.152514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7fc1-0834-4e69-bd70-01c6d15b2a52 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.164885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7fc1-0834-4e69-bd70-01c6d15b2a52 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.169772 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218c673-7460-4b0c-a9f2-7980572dd96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.170518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218c673-7460-4b0c-a9f2-7980572dd96a admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.170722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218c673-7460-4b0c-a9f2-7980572dd96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.171006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218c673-7460-4b0c-a9f2-7980572dd96a admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.171698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218c673-7460-4b0c-a9f2-7980572dd96a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.173752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218c673-7460-4b0c-a9f2-7980572dd96a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.195355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '229277f47d26424fab98e71d968cf35f', 'request_id': 'req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.202427 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '229277f47d26424fab98e71d968cf35f', 'request_id': 'req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.211016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1019/2028] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:26 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:26.212016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '229277f47d26424fab98e71d968cf35f', 'request_id': 'req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.212305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218c673-7460-4b0c-a9f2-7980572dd96a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.224382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] The token cache is being invalidate because project 229277f47d26424fab98e71d968cf35f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:26.224677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.225388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.227017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1010/2029] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:26 2022] DELETE /identity/v3/projects/229277f47d26424fab98e71d968cf35f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:26.227861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.229270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.243513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:26.277618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218c673-7460-4b0c-a9f2-7980572dd96a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.282381 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.283270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.283660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.284004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] PATH_INFO: `/v3/users/250fc7108f0d44fb8e98c3c16e0296a7` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.285264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=250fc7108f0d44fb8e98c3c16e0296a7)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.287189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.294912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d1af2-58cd-46bb-ae07-2bf0b7cb7563 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.300289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df51f9-6b30-457f-8fe6-059f875e4227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.300972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df51f9-6b30-457f-8fe6-059f875e4227 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.301252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df51f9-6b30-457f-8fe6-059f875e4227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.301525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df51f9-6b30-457f-8fe6-059f875e4227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.302259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df51f9-6b30-457f-8fe6-059f875e4227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.305191 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df51f9-6b30-457f-8fe6-059f875e4227 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.319379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1011/2030] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:26.320372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df51f9-6b30-457f-8fe6-059f875e4227 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.322233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-56df51f9-6b30-457f-8fe6-059f875e4227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.338146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df51f9-6b30-457f-8fe6-059f875e4227 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.343133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0eaca2-30ab-4ee7-be72-cec625ce7ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.343810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0eaca2-30ab-4ee7-be72-cec625ce7ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.344236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0eaca2-30ab-4ee7-be72-cec625ce7ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.344608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0eaca2-30ab-4ee7-be72-cec625ce7ba8 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.345303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0eaca2-30ab-4ee7-be72-cec625ce7ba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.347191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '250fc7108f0d44fb8e98c3c16e0296a7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.347411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0eaca2-30ab-4ee7-be72-cec625ce7ba8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.354758 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '250fc7108f0d44fb8e98c3c16e0296a7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.355467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '250fc7108f0d44fb8e98c3c16e0296a7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.365593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '250fc7108f0d44fb8e98c3c16e0296a7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.370563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1012/2031] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:26 2022] POST /identity/v3/users => generated 298 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:26.371647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0eaca2-30ab-4ee7-be72-cec625ce7ba8 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.375204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0c0eaca2-30ab-4ee7-be72-cec625ce7ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.381808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '250fc7108f0d44fb8e98c3c16e0296a7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.384033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:26.389142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1020/2032] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:26 2022] DELETE /identity/v3/users/250fc7108f0d44fb8e98c3c16e0296a7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:26.389709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0eaca2-30ab-4ee7-be72-cec625ce7ba8 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.389820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.391578 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.454859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.456334 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.456759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.457205 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] PATH_INFO: `/v3/users/bbf0fd1e96904f939d50996e1143b31d` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.458767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbf0fd1e96904f939d50996e1143b31d)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.460874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.465738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585dc07-f3f2-4cb7-9393-767260094ae9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.471653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e015566-6724-4674-81b0-c4a2de2c9b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.473894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e015566-6724-4674-81b0-c4a2de2c9b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.474271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e015566-6724-4674-81b0-c4a2de2c9b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.474654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e015566-6724-4674-81b0-c4a2de2c9b33 admin admin] PATH_INFO: `/v3/projects/84489b2c376245489f0af5b153e2c860/users/cfa3a3b2f3d6493aa4de5842c0c86c71/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.492340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e015566-6724-4674-81b0-c4a2de2c9b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84489b2c376245489f0af5b153e2c860, user_id=cfa3a3b2f3d6493aa4de5842c0c86c71, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.495023 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e015566-6724-4674-81b0-c4a2de2c9b33 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.511738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1021/2033] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:26 2022] PUT /identity/v3/projects/84489b2c376245489f0af5b153e2c860/users/cfa3a3b2f3d6493aa4de5842c0c86c71/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:26.513065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e015566-6724-4674-81b0-c4a2de2c9b33 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.514748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbf0fd1e96904f939d50996e1143b31d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.515201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9e015566-6724-4674-81b0-c4a2de2c9b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.515406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbf0fd1e96904f939d50996e1143b31d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.523736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbf0fd1e96904f939d50996e1143b31d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.524540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbf0fd1e96904f939d50996e1143b31d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.539255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fbe0d0de-41fb-47e7-a166-443ff5e6efb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbf0fd1e96904f939d50996e1143b31d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:26.559003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1013/2034] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:26 2022] DELETE /identity/v3/users/bbf0fd1e96904f939d50996e1143b31d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:26.560085 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb03e84-3359-4fce-b52a-15a8b20545e9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.560395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb03e84-3359-4fce-b52a-15a8b20545e9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.560777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb03e84-3359-4fce-b52a-15a8b20545e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.586155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-abb03e84-3359-4fce-b52a-15a8b20545e9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:26.587923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-abb03e84-3359-4fce-b52a-15a8b20545e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.596340 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:26.634470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e015566-6724-4674-81b0-c4a2de2c9b33 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.697884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1014/2035] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:26.699408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397ff19-147f-448a-87e7-50dd7f6701b6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.699694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397ff19-147f-448a-87e7-50dd7f6701b6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.699962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397ff19-147f-448a-87e7-50dd7f6701b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.715656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfc6af-e34e-4ebd-be4e-735af24b0ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.716628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfc6af-e34e-4ebd-be4e-735af24b0ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.717050 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfc6af-e34e-4ebd-be4e-735af24b0ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.717461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfc6af-e34e-4ebd-be4e-735af24b0ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.718334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfc6af-e34e-4ebd-be4e-735af24b0ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.721380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfc6af-e34e-4ebd-be4e-735af24b0ed1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.728654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f397ff19-147f-448a-87e7-50dd7f6701b6 None None] MFA Rules not processed for user `cfa3a3b2f3d6493aa4de5842c0c86c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:26.729887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f397ff19-147f-448a-87e7-50dd7f6701b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.733617 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1022/2036] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:26.737850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfc6af-e34e-4ebd-be4e-735af24b0ed1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.739605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-68bfc6af-e34e-4ebd-be4e-735af24b0ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.753878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfc6af-e34e-4ebd-be4e-735af24b0ed1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.760167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76849a41-9894-44d9-8d2a-e0fed2329c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.761384 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76849a41-9894-44d9-8d2a-e0fed2329c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.761675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76849a41-9894-44d9-8d2a-e0fed2329c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.761952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76849a41-9894-44d9-8d2a-e0fed2329c79 admin admin] PATH_INFO: `/v3/projects/fa3f715ae9d0465ba3b90a8d0cd13bf2/users/598a4e01921b40548fff2ded1148fbef/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.781619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76849a41-9894-44d9-8d2a-e0fed2329c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa3f715ae9d0465ba3b90a8d0cd13bf2, user_id=598a4e01921b40548fff2ded1148fbef, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.785248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76849a41-9894-44d9-8d2a-e0fed2329c79 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.802032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1023/2037] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:26 2022] PUT /identity/v3/projects/fa3f715ae9d0465ba3b90a8d0cd13bf2/users/598a4e01921b40548fff2ded1148fbef/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:26.805968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76849a41-9894-44d9-8d2a-e0fed2329c79 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.806844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1015/2038] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:26.811034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397ff19-147f-448a-87e7-50dd7f6701b6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.812422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f397ff19-147f-448a-87e7-50dd7f6701b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:26.814894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:26.827127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397ff19-147f-448a-87e7-50dd7f6701b6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.887397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76849a41-9894-44d9-8d2a-e0fed2329c79 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.887621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4086ccc-d6ee-4544-8b11-e6cd15413eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.888473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4086ccc-d6ee-4544-8b11-e6cd15413eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.888791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4086ccc-d6ee-4544-8b11-e6cd15413eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.889127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4086ccc-d6ee-4544-8b11-e6cd15413eb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.889902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4086ccc-d6ee-4544-8b11-e6cd15413eb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.891866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4086ccc-d6ee-4544-8b11-e6cd15413eb1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.892616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f93b8-6434-480e-9d84-41787baae0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.893436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f93b8-6434-480e-9d84-41787baae0eb admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.893927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f93b8-6434-480e-9d84-41787baae0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.894264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f93b8-6434-480e-9d84-41787baae0eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.895073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f93b8-6434-480e-9d84-41787baae0eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:26.896788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f93b8-6434-480e-9d84-41787baae0eb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:26.907760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1024/2039] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:26.913017 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f93b8-6434-480e-9d84-41787baae0eb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.923827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1016/2040] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:26 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:26.925721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4086ccc-d6ee-4544-8b11-e6cd15413eb1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:26.926162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f93b8-6434-480e-9d84-41787baae0eb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:26.971689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:26.993971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f5f63-e19a-4951-b7ea-c4a08b6a8469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:26.994953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f5f63-e19a-4951-b7ea-c4a08b6a8469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:26.995231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f5f63-e19a-4951-b7ea-c4a08b6a8469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:26.995500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f5f63-e19a-4951-b7ea-c4a08b6a8469 admin admin] PATH_INFO: `/v3/projects/fa3f715ae9d0465ba3b90a8d0cd13bf2/users/598a4e01921b40548fff2ded1148fbef/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:26.997754 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f5f63-e19a-4951-b7ea-c4a08b6a8469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa3f715ae9d0465ba3b90a8d0cd13bf2, user_id=598a4e01921b40548fff2ded1148fbef, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:27.000486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f5f63-e19a-4951-b7ea-c4a08b6a8469 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:27.011343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4086ccc-d6ee-4544-8b11-e6cd15413eb1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:27.012963 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1025/2041] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:26 2022] PUT /identity/v3/projects/fa3f715ae9d0465ba3b90a8d0cd13bf2/users/598a4e01921b40548fff2ded1148fbef/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:27.014079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f5f63-e19a-4951-b7ea-c4a08b6a8469 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:27.016180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-165f5f63-e19a-4951-b7ea-c4a08b6a8469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:27.112366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c087c3-815d-4894-a660-3e01deb20a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:27.113141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c087c3-815d-4894-a660-3e01deb20a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.113557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c087c3-815d-4894-a660-3e01deb20a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.113884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c087c3-815d-4894-a660-3e01deb20a8e admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.114785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c087c3-815d-4894-a660-3e01deb20a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:27.117303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c087c3-815d-4894-a660-3e01deb20a8e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:27.131846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f5f63-e19a-4951-b7ea-c4a08b6a8469 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:27.136249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6859966-0dfc-4506-a1b9-17cb6b4f1bdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:27.136972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859966-0dfc-4506-a1b9-17cb6b4f1bdc service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.137252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859966-0dfc-4506-a1b9-17cb6b4f1bdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.137524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859966-0dfc-4506-a1b9-17cb6b4f1bdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.141249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1017/2042] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:26 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:27.145689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa801fa-3220-4daa-adb0-8dd97b6576b7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.146049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa801fa-3220-4daa-adb0-8dd97b6576b7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.146962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa801fa-3220-4daa-adb0-8dd97b6576b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.148735 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6859966-0dfc-4506-a1b9-17cb6b4f1bdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:27.150179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6859966-0dfc-4506-a1b9-17cb6b4f1bdc service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:27.164033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1026/2043] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:27.165036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6859966-0dfc-4506-a1b9-17cb6b4f1bdc service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:27.177040 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6859966-0dfc-4506-a1b9-17cb6b4f1bdc service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:27.178543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3fa801fa-3220-4daa-adb0-8dd97b6576b7 None None] MFA Rules not processed for user `598a4e01921b40548fff2ded1148fbef`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:27.180034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3fa801fa-3220-4daa-adb0-8dd97b6576b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:27.182560 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efad6a-afc7-4703-8c3d-95efb75f5689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:27.183180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efad6a-afc7-4703-8c3d-95efb75f5689 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.183618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efad6a-afc7-4703-8c3d-95efb75f5689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.184148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efad6a-afc7-4703-8c3d-95efb75f5689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.185148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efad6a-afc7-4703-8c3d-95efb75f5689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:27.187255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efad6a-afc7-4703-8c3d-95efb75f5689 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:27.203241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1027/2044] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:27.209072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efad6a-afc7-4703-8c3d-95efb75f5689 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:27.230402 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efad6a-afc7-4703-8c3d-95efb75f5689 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:27.246600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21835ed0-c08c-4796-9514-3eedf04e9bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:27.249197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21835ed0-c08c-4796-9514-3eedf04e9bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.250613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21835ed0-c08c-4796-9514-3eedf04e9bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.253330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21835ed0-c08c-4796-9514-3eedf04e9bf5 admin admin] PATH_INFO: `/v3/projects/d8d54e6d6142497981b69b989ca6d131/users/d6b44d46dcc24ad78012de8ebe9ce3f1/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.283204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21835ed0-c08c-4796-9514-3eedf04e9bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d54e6d6142497981b69b989ca6d131, user_id=d6b44d46dcc24ad78012de8ebe9ce3f1, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:27.286497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21835ed0-c08c-4796-9514-3eedf04e9bf5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:27.301818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:27.303273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1018/2045] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:27.318920 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa801fa-3220-4daa-adb0-8dd97b6576b7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:27.320764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3fa801fa-3220-4daa-adb0-8dd97b6576b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:27.321395 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1028/2046] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:27 2022] PUT /identity/v3/projects/d8d54e6d6142497981b69b989ca6d131/users/d6b44d46dcc24ad78012de8ebe9ce3f1/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:27.325001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21835ed0-c08c-4796-9514-3eedf04e9bf5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:27.417760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:27.426237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa801fa-3220-4daa-adb0-8dd97b6576b7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:27.463274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21835ed0-c08c-4796-9514-3eedf04e9bf5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:27.469227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b297a-fd5f-4335-b2c3-ae3b6aa6eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:27.469919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b297a-fd5f-4335-b2c3-ae3b6aa6eea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.470202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b297a-fd5f-4335-b2c3-ae3b6aa6eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.470525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b297a-fd5f-4335-b2c3-ae3b6aa6eea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.482942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b297a-fd5f-4335-b2c3-ae3b6aa6eea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:27.484839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b297a-fd5f-4335-b2c3-ae3b6aa6eea6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:27.510637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1029/2047] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:27.515645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b297a-fd5f-4335-b2c3-ae3b6aa6eea6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:27.550081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b297a-fd5f-4335-b2c3-ae3b6aa6eea6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:27.555307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce98cc-9c33-4826-9104-0228490bafd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:27.557469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce98cc-9c33-4826-9104-0228490bafd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.557919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce98cc-9c33-4826-9104-0228490bafd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.558346 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce98cc-9c33-4826-9104-0228490bafd4 admin admin] PATH_INFO: `/v3/projects/d8d54e6d6142497981b69b989ca6d131/users/d6b44d46dcc24ad78012de8ebe9ce3f1/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.560592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce98cc-9c33-4826-9104-0228490bafd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d54e6d6142497981b69b989ca6d131, user_id=d6b44d46dcc24ad78012de8ebe9ce3f1, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:27.563752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce98cc-9c33-4826-9104-0228490bafd4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:27.569058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb6ce3-d71e-4ebf-b85a-7a56c3080613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:27.573500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb6ce3-d71e-4ebf-b85a-7a56c3080613 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.573792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb6ce3-d71e-4ebf-b85a-7a56c3080613 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.574105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb6ce3-d71e-4ebf-b85a-7a56c3080613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.576593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7efb6ce3-d71e-4ebf-b85a-7a56c3080613 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:27.604440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1030/2048] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:27 2022] PUT /identity/v3/projects/d8d54e6d6142497981b69b989ca6d131/users/d6b44d46dcc24ad78012de8ebe9ce3f1/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:27.605851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce98cc-9c33-4826-9104-0228490bafd4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:27.608892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b3ce98cc-9c33-4826-9104-0228490bafd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:27.699843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce98cc-9c33-4826-9104-0228490bafd4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:27.701747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb6ce3-d71e-4ebf-b85a-7a56c3080613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:27.706488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:27.707142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb6ce3-d71e-4ebf-b85a-7a56c3080613 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:27.707418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.707815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.708196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] PATH_INFO: `/v3/projects/7058801184ba4f94a7b6ee0cccadccb4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.709521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7058801184ba4f94a7b6ee0cccadccb4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:27.711878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:27.744164 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1019/2049] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:27.752517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93508a2f-a5ca-4632-86cf-2f37a4bb4d1c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.752750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93508a2f-a5ca-4632-86cf-2f37a4bb4d1c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.753069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93508a2f-a5ca-4632-86cf-2f37a4bb4d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.779797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7058801184ba4f94a7b6ee0cccadccb4', 'request_id': 'req-ac505b86-9f0d-4b87-8853-bd42d1569854'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:27.794896 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7058801184ba4f94a7b6ee0cccadccb4', 'request_id': 'req-ac505b86-9f0d-4b87-8853-bd42d1569854'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:27.798675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-93508a2f-a5ca-4632-86cf-2f37a4bb4d1c None None] MFA Rules not processed for user `d6b44d46dcc24ad78012de8ebe9ce3f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:27.800060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-93508a2f-a5ca-4632-86cf-2f37a4bb4d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:27.814093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7058801184ba4f94a7b6ee0cccadccb4', 'request_id': 'req-ac505b86-9f0d-4b87-8853-bd42d1569854'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:27.841044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] The token cache is being invalidate because project 7058801184ba4f94a7b6ee0cccadccb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:27.841611 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:27.845137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:27.849687 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1031/2050] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:27 2022] DELETE /identity/v3/projects/7058801184ba4f94a7b6ee0cccadccb4 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:27.852755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:27.855207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:27.896945 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:27.948144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1020/2051] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:27.949639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5b567-123d-4281-b0d6-5c55b37076d3 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:27.949927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5b567-123d-4281-b0d6-5c55b37076d3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:27.958930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5b567-123d-4281-b0d6-5c55b37076d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:27.979493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-aee5b567-123d-4281-b0d6-5c55b37076d3 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:27.980980 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-aee5b567-123d-4281-b0d6-5c55b37076d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:27.990865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1021/2052] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:27.991719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5b567-123d-4281-b0d6-5c55b37076d3 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:27.994682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac505b86-9f0d-4b87-8853-bd42d1569854 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:27.999043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-aee5b567-123d-4281-b0d6-5c55b37076d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:28.000769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.002256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.002895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.003621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] PATH_INFO: `/v3/projects/a250614940da4b42aaf0477dd64ff0f3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.005460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a250614940da4b42aaf0477dd64ff0f3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.007477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.044788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:28.064630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5b567-123d-4281-b0d6-5c55b37076d3 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.071681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a250614940da4b42aaf0477dd64ff0f3', 'request_id': 'req-f9118371-3789-431a-ad35-bbf2864e70a5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:28.089960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a250614940da4b42aaf0477dd64ff0f3', 'request_id': 'req-f9118371-3789-431a-ad35-bbf2864e70a5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:28.110432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a250614940da4b42aaf0477dd64ff0f3', 'request_id': 'req-f9118371-3789-431a-ad35-bbf2864e70a5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:28.134698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] The token cache is being invalidate because project a250614940da4b42aaf0477dd64ff0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:28.135096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:28.137057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9118371-3789-431a-ad35-bbf2864e70a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:28.140594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1032/2053] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:27 2022] DELETE /identity/v3/projects/a250614940da4b42aaf0477dd64ff0f3 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:28.141922 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170293ca-98f4-4469-a0f7-6a824faf4abc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.142329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170293ca-98f4-4469-a0f7-6a824faf4abc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.142698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170293ca-98f4-4469-a0f7-6a824faf4abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.187484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-170293ca-98f4-4469-a0f7-6a824faf4abc None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:28.190607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-170293ca-98f4-4469-a0f7-6a824faf4abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:28.204451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12412582-8db9-4930-b6a1-266168aa801c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.205129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12412582-8db9-4930-b6a1-266168aa801c service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.205859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12412582-8db9-4930-b6a1-266168aa801c service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.206150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12412582-8db9-4930-b6a1-266168aa801c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.215936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-12412582-8db9-4930-b6a1-266168aa801c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:28.335847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1033/2054] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:28.339942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc7855-ca33-480e-b8c7-88186f3230bb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.340343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc7855-ca33-480e-b8c7-88186f3230bb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.341345 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc7855-ca33-480e-b8c7-88186f3230bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.380033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8cdc7855-ca33-480e-b8c7-88186f3230bb None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:28.381392 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12412582-8db9-4930-b6a1-266168aa801c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.381580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8cdc7855-ca33-480e-b8c7-88186f3230bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:28.384322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12412582-8db9-4930-b6a1-266168aa801c service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.397269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1034/2055] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:28.397269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc7855-ca33-480e-b8c7-88186f3230bb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.403744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc7855-ca33-480e-b8c7-88186f3230bb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.409816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c02eb-f77e-43d0-8bb8-00eb50674e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.410534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c02eb-f77e-43d0-8bb8-00eb50674e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.410840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c02eb-f77e-43d0-8bb8-00eb50674e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.411111 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c02eb-f77e-43d0-8bb8-00eb50674e53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.411894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c02eb-f77e-43d0-8bb8-00eb50674e53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:28.412250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c02eb-f77e-43d0-8bb8-00eb50674e53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.413458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1022/2056] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:28.414930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece07f9b-2917-45d5-b0f6-99868b18fc95 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.415118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece07f9b-2917-45d5-b0f6-99868b18fc95 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.415395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece07f9b-2917-45d5-b0f6-99868b18fc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.423349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c02eb-f77e-43d0-8bb8-00eb50674e53 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.436189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ece07f9b-2917-45d5-b0f6-99868b18fc95 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:28.437448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ece07f9b-2917-45d5-b0f6-99868b18fc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:28.443945 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1035/2057] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:28.444932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c02eb-f77e-43d0-8bb8-00eb50674e53 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.455278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c02eb-f77e-43d0-8bb8-00eb50674e53 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.457378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1023/2058] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:28.462028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece07f9b-2917-45d5-b0f6-99868b18fc95 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.463908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f15db-0c26-43ea-afce-e9ca11ed88c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.464724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f15db-0c26-43ea-afce-e9ca11ed88c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.465009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f15db-0c26-43ea-afce-e9ca11ed88c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.465284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f15db-0c26-43ea-afce-e9ca11ed88c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.465995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f15db-0c26-43ea-afce-e9ca11ed88c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.468330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f15db-0c26-43ea-afce-e9ca11ed88c2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.473061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece07f9b-2917-45d5-b0f6-99868b18fc95 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.477656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e97e51-53b7-4fb8-ad48-181888310b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.478330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e97e51-53b7-4fb8-ad48-181888310b70 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.478613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e97e51-53b7-4fb8-ad48-181888310b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.478884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e97e51-53b7-4fb8-ad48-181888310b70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.479651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e97e51-53b7-4fb8-ad48-181888310b70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:28.480495 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e97e51-53b7-4fb8-ad48-181888310b70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.481601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e97e51-53b7-4fb8-ad48-181888310b70 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.500133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1024/2059] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:28.501607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e97e51-53b7-4fb8-ad48-181888310b70 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.512059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1036/2060] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:28 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:28.512251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e97e51-53b7-4fb8-ad48-181888310b70 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.513325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f15db-0c26-43ea-afce-e9ca11ed88c2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.520043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:28.588637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502f1c7-2761-4c40-84d5-050181d1ef23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.589474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502f1c7-2761-4c40-84d5-050181d1ef23 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.589758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502f1c7-2761-4c40-84d5-050181d1ef23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.590030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502f1c7-2761-4c40-84d5-050181d1ef23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.590902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502f1c7-2761-4c40-84d5-050181d1ef23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.592852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502f1c7-2761-4c40-84d5-050181d1ef23 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.600015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f15db-0c26-43ea-afce-e9ca11ed88c2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.607779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b7797-e446-4875-8055-5970238d69c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.609189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b7797-e446-4875-8055-5970238d69c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.609838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b7797-e446-4875-8055-5970238d69c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.610048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b7797-e446-4875-8055-5970238d69c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.610998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b7797-e446-4875-8055-5970238d69c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.613875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b7797-e446-4875-8055-5970238d69c8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.637927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1037/2061] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:28 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:28.643208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1025/2062] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:28 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:28.643461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b7797-e446-4875-8055-5970238d69c8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.649561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502f1c7-2761-4c40-84d5-050181d1ef23 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.652204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b7797-e446-4875-8055-5970238d69c8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.662536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:28.725042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda6e83-e360-4a56-96b7-916ddd3f3eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.726641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6e83-e360-4a56-96b7-916ddd3f3eed admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.727200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6e83-e360-4a56-96b7-916ddd3f3eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.727892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6e83-e360-4a56-96b7-916ddd3f3eed admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.729414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda6e83-e360-4a56-96b7-916ddd3f3eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.731786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda6e83-e360-4a56-96b7-916ddd3f3eed admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.739070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502f1c7-2761-4c40-84d5-050181d1ef23 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.748978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5929dc7-1744-4b9a-9fe9-9aa78b590bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.748978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5929dc7-1744-4b9a-9fe9-9aa78b590bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.748978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5929dc7-1744-4b9a-9fe9-9aa78b590bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.748978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5929dc7-1744-4b9a-9fe9-9aa78b590bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.749451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5929dc7-1744-4b9a-9fe9-9aa78b590bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.750232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1038/2063] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:28.751301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5929dc7-1744-4b9a-9fe9-9aa78b590bb6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.761684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda6e83-e360-4a56-96b7-916ddd3f3eed admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.774873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda6e83-e360-4a56-96b7-916ddd3f3eed admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.780597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1026/2064] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:28 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:28.785560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5929dc7-1744-4b9a-9fe9-9aa78b590bb6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.796641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c08411-7878-4162-baff-6a922a2e3e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.797108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5929dc7-1744-4b9a-9fe9-9aa78b590bb6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.797333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c08411-7878-4162-baff-6a922a2e3e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.797615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c08411-7878-4162-baff-6a922a2e3e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.797890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c08411-7878-4162-baff-6a922a2e3e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.798628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c08411-7878-4162-baff-6a922a2e3e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.800387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c08411-7878-4162-baff-6a922a2e3e82 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.802899 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fea98-886b-4860-b9f4-78de642f8f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.803575 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fea98-886b-4860-b9f4-78de642f8f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.803859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fea98-886b-4860-b9f4-78de642f8f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.804215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fea98-886b-4860-b9f4-78de642f8f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.805103 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fea98-886b-4860-b9f4-78de642f8f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.807298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fea98-886b-4860-b9f4-78de642f8f68 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.812638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1039/2065] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:28.817390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c08411-7878-4162-baff-6a922a2e3e82 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.831698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1027/2066] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:28.832744 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fea98-886b-4860-b9f4-78de642f8f68 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.841197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c08411-7878-4162-baff-6a922a2e3e82 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.845293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fea98-886b-4860-b9f4-78de642f8f68 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:28.851755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d958f-6ea1-4266-afaa-c8d03dc4b673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.852481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d958f-6ea1-4266-afaa-c8d03dc4b673 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.852786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d958f-6ea1-4266-afaa-c8d03dc4b673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.853033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d958f-6ea1-4266-afaa-c8d03dc4b673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.853773 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d958f-6ea1-4266-afaa-c8d03dc4b673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.854542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3acb8-26b2-4e1d-914c-3cf33cbb0768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:28.856637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3acb8-26b2-4e1d-914c-3cf33cbb0768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.856927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3acb8-26b2-4e1d-914c-3cf33cbb0768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.857202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3acb8-26b2-4e1d-914c-3cf33cbb0768 admin admin] PATH_INFO: `/v3/projects/1a3c0d48f55c4b6cb4a7e5d772cd18a2/users/96025df19bb64985908fc5187af44888/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:28.864862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d958f-6ea1-4266-afaa-c8d03dc4b673 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.883544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3acb8-26b2-4e1d-914c-3cf33cbb0768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a3c0d48f55c4b6cb4a7e5d772cd18a2, user_id=96025df19bb64985908fc5187af44888, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:28.886747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3acb8-26b2-4e1d-914c-3cf33cbb0768 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:28.907563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1028/2067] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:28.910471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1040/2068] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:28 2022] PUT /identity/v3/projects/1a3c0d48f55c4b6cb4a7e5d772cd18a2/users/96025df19bb64985908fc5187af44888/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:28.913338 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3acb8-26b2-4e1d-914c-3cf33cbb0768 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:28.921611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee8e3e-0b18-4655-98eb-c8ef876dc121 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:28.921912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee8e3e-0b18-4655-98eb-c8ef876dc121 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:28.922180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee8e3e-0b18-4655-98eb-c8ef876dc121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.000430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:29.018736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-faee8e3e-0b18-4655-98eb-c8ef876dc121 None None] MFA Rules not processed for user `96025df19bb64985908fc5187af44888`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:29.019311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3acb8-26b2-4e1d-914c-3cf33cbb0768 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.019994 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-faee8e3e-0b18-4655-98eb-c8ef876dc121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:29.028958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35823e20-ede9-48a1-b251-9dbdefd83b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:29.030279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35823e20-ede9-48a1-b251-9dbdefd83b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.031034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35823e20-ede9-48a1-b251-9dbdefd83b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.031319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35823e20-ede9-48a1-b251-9dbdefd83b7e admin admin] PATH_INFO: `/v3/projects/0c934c24022c47ae9c73389fffe022fa/users/c56e3bcc509f4d4abc860b954be44380/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.059056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35823e20-ede9-48a1-b251-9dbdefd83b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c934c24022c47ae9c73389fffe022fa, user_id=c56e3bcc509f4d4abc860b954be44380, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:29.063154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35823e20-ede9-48a1-b251-9dbdefd83b7e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:29.087486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1041/2069] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:28 2022] PUT /identity/v3/projects/0c934c24022c47ae9c73389fffe022fa/users/c56e3bcc509f4d4abc860b954be44380/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:29.088824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35823e20-ede9-48a1-b251-9dbdefd83b7e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:29.090911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-35823e20-ede9-48a1-b251-9dbdefd83b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:29.134065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1029/2070] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:29.135997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0ca71-c659-4acd-9b5b-d963f0a9207d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.136510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0ca71-c659-4acd-9b5b-d963f0a9207d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.136975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0ca71-c659-4acd-9b5b-d963f0a9207d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.172253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35823e20-ede9-48a1-b251-9dbdefd83b7e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.172862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e5d0ca71-c659-4acd-9b5b-d963f0a9207d None None] MFA Rules not processed for user `c56e3bcc509f4d4abc860b954be44380`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:29.174211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e5d0ca71-c659-4acd-9b5b-d963f0a9207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:29.178263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:29.179259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.179699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.179990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] PATH_INFO: `/v3/users/cfa3a3b2f3d6493aa4de5842c0c86c71` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.181315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa3a3b2f3d6493aa4de5842c0c86c71)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:29.184090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:29.286266 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa3a3b2f3d6493aa4de5842c0c86c71'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:29.289171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:29.294388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa3a3b2f3d6493aa4de5842c0c86c71'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:29.295229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa3a3b2f3d6493aa4de5842c0c86c71'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:29.304221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1030/2071] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:29.305095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0ca71-c659-4acd-9b5b-d963f0a9207d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:29.313475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa3a3b2f3d6493aa4de5842c0c86c71'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:29.318651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0ca71-c659-4acd-9b5b-d963f0a9207d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.324087 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f4845-5eab-441b-8b43-320021519f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:29.324932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f4845-5eab-441b-8b43-320021519f86 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.325299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f4845-5eab-441b-8b43-320021519f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.325721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f4845-5eab-441b-8b43-320021519f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.326746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f4845-5eab-441b-8b43-320021519f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:29.327672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa3a3b2f3d6493aa4de5842c0c86c71'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:29.328969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f4845-5eab-441b-8b43-320021519f86 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:29.335609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1042/2072] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:29 2022] DELETE /identity/v3/users/cfa3a3b2f3d6493aa4de5842c0c86c71 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:29.336332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:29.337749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:29.365946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1031/2073] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:29 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:29.367619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf854974-f5d2-4a04-9e51-46555792af9f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.368895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf854974-f5d2-4a04-9e51-46555792af9f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.369599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf854974-f5d2-4a04-9e51-46555792af9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.419127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cf854974-f5d2-4a04-9e51-46555792af9f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:29.420403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cf854974-f5d2-4a04-9e51-46555792af9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:29.422775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:29.430357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d75e6-7718-4e12-9f0f-6a01a5e6e2ba admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.504930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1032/2074] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:29.509188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf854974-f5d2-4a04-9e51-46555792af9f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:29.510901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cf854974-f5d2-4a04-9e51-46555792af9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:29.525942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf854974-f5d2-4a04-9e51-46555792af9f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.536239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf79723-f719-499b-89b9-6fd325aace87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:29.536941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf79723-f719-499b-89b9-6fd325aace87 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.537723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf79723-f719-499b-89b9-6fd325aace87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.538006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf79723-f719-499b-89b9-6fd325aace87 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.539261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf79723-f719-499b-89b9-6fd325aace87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:29.543823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf79723-f719-499b-89b9-6fd325aace87 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:29.584370 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a00e2c-a573-4da3-849b-7dff41c94d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:29.584531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a00e2c-a573-4da3-849b-7dff41c94d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.584795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a00e2c-a573-4da3-849b-7dff41c94d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.585055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a00e2c-a573-4da3-849b-7dff41c94d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.585829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a00e2c-a573-4da3-849b-7dff41c94d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:29.588192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a00e2c-a573-4da3-849b-7dff41c94d2f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:29.604685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1033/2075] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:29 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:29.605733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf79723-f719-499b-89b9-6fd325aace87 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:29.608679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9cf79723-f719-499b-89b9-6fd325aace87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:29.641851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1043/2076] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:29 2022] POST /identity/v3/projects => generated 375 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:29.643797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a00e2c-a573-4da3-849b-7dff41c94d2f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:29.692296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf79723-f719-499b-89b9-6fd325aace87 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.785309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a00e2c-a573-4da3-849b-7dff41c94d2f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.791056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250916f6-353d-4af9-b7fa-a23f0573c49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:29.791743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250916f6-353d-4af9-b7fa-a23f0573c49a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.792022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250916f6-353d-4af9-b7fa-a23f0573c49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.792368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250916f6-353d-4af9-b7fa-a23f0573c49a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.793151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250916f6-353d-4af9-b7fa-a23f0573c49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:29.793644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7687afc-32f1-445c-9e76-81ecc023b043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:29.794536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7687afc-32f1-445c-9e76-81ecc023b043 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.794826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250916f6-353d-4af9-b7fa-a23f0573c49a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:29.795252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7687afc-32f1-445c-9e76-81ecc023b043 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.795702 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7687afc-32f1-445c-9e76-81ecc023b043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.807692 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1044/2077] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:29.808807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250916f6-353d-4af9-b7fa-a23f0573c49a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:29.809564 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7687afc-32f1-445c-9e76-81ecc023b043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:29.811393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7687afc-32f1-445c-9e76-81ecc023b043 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:29.823550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1034/2078] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:29.824691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7687afc-32f1-445c-9e76-81ecc023b043 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:29.832285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250916f6-353d-4af9-b7fa-a23f0573c49a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.834577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7687afc-32f1-445c-9e76-81ecc023b043 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.843490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178d19c-b04c-483e-a538-682a6ceae2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:29.844241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178d19c-b04c-483e-a538-682a6ceae2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.845000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178d19c-b04c-483e-a538-682a6ceae2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.845000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178d19c-b04c-483e-a538-682a6ceae2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.845704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178d19c-b04c-483e-a538-682a6ceae2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:29.848099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178d19c-b04c-483e-a538-682a6ceae2a5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:29.850632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927429d-a124-49f4-b5a7-3662e325ae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:29.852635 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927429d-a124-49f4-b5a7-3662e325ae47 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.853524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927429d-a124-49f4-b5a7-3662e325ae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.858188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927429d-a124-49f4-b5a7-3662e325ae47 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.858188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927429d-a124-49f4-b5a7-3662e325ae47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:29.861760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1035/2079] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:29.864294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927429d-a124-49f4-b5a7-3662e325ae47 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:29.866020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178d19c-b04c-483e-a538-682a6ceae2a5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:29.875039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:29.880227 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178d19c-b04c-483e-a538-682a6ceae2a5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.889139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6de89-3232-4ff5-8d1f-cf4d560ae7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:29.889956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6de89-3232-4ff5-8d1f-cf4d560ae7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.890302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6de89-3232-4ff5-8d1f-cf4d560ae7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.890599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6de89-3232-4ff5-8d1f-cf4d560ae7c1 admin admin] PATH_INFO: `/v3/projects/8c5997e318844d67beb7afef7ea86e9e/users/0bfc852deaaf41a98919cc338be2e1d7/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.903364 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1045/2080] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:29 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:29.907343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927429d-a124-49f4-b5a7-3662e325ae47 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:29.913838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6de89-3232-4ff5-8d1f-cf4d560ae7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5997e318844d67beb7afef7ea86e9e, user_id=0bfc852deaaf41a98919cc338be2e1d7, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:29.916777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6de89-3232-4ff5-8d1f-cf4d560ae7c1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:29.931702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927429d-a124-49f4-b5a7-3662e325ae47 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:29.940299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1036/2081] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:29 2022] PUT /identity/v3/projects/8c5997e318844d67beb7afef7ea86e9e/users/0bfc852deaaf41a98919cc338be2e1d7/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:29.950197 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bce02-ff32-4f87-abc9-845fe64ef53a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:29.951164 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bce02-ff32-4f87-abc9-845fe64ef53a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:29.951642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bce02-ff32-4f87-abc9-845fe64ef53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:29.987398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5c4bce02-ff32-4f87-abc9-845fe64ef53a None None] MFA Rules not processed for user `0bfc852deaaf41a98919cc338be2e1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:29.989455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5c4bce02-ff32-4f87-abc9-845fe64ef53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:30.012561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab7458-4eec-41c6-9e04-decff6067862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.013331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab7458-4eec-41c6-9e04-decff6067862 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.013686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab7458-4eec-41c6-9e04-decff6067862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.014071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab7458-4eec-41c6-9e04-decff6067862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.014931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab7458-4eec-41c6-9e04-decff6067862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.017298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab7458-4eec-41c6-9e04-decff6067862 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.030982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1046/2082] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:30.032241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab7458-4eec-41c6-9e04-decff6067862 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.039995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:30.059586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab7458-4eec-41c6-9e04-decff6067862 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.068038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179140b-aca2-4118-8752-bd75e1f3d9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.068815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179140b-aca2-4118-8752-bd75e1f3d9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.069102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179140b-aca2-4118-8752-bd75e1f3d9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.069710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179140b-aca2-4118-8752-bd75e1f3d9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.070606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179140b-aca2-4118-8752-bd75e1f3d9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.072690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179140b-aca2-4118-8752-bd75e1f3d9d3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.089755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1047/2083] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:30.093030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179140b-aca2-4118-8752-bd75e1f3d9d3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.107602 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179140b-aca2-4118-8752-bd75e1f3d9d3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.108888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1037/2084] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:30.113319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bce02-ff32-4f87-abc9-845fe64ef53a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.116235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa9928-0dd3-4255-95fb-24540b0a673c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.117153 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa9928-0dd3-4255-95fb-24540b0a673c admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.117438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa9928-0dd3-4255-95fb-24540b0a673c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.117717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa9928-0dd3-4255-95fb-24540b0a673c admin admin] PATH_INFO: `/v3/projects/b9ebb7f483a84ea5beab691ce0144060/users/9f95b306df78477d9521d5304195ce69/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.124835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bce02-ff32-4f87-abc9-845fe64ef53a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.129834 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505c723-54f3-4b61-9435-f18e0ad82f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.130506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505c723-54f3-4b61-9435-f18e0ad82f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.130785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505c723-54f3-4b61-9435-f18e0ad82f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.131054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505c723-54f3-4b61-9435-f18e0ad82f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.131751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505c723-54f3-4b61-9435-f18e0ad82f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.133709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505c723-54f3-4b61-9435-f18e0ad82f58 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.135272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa9928-0dd3-4255-95fb-24540b0a673c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9ebb7f483a84ea5beab691ce0144060, user_id=9f95b306df78477d9521d5304195ce69, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.138316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa9928-0dd3-4255-95fb-24540b0a673c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.168098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1048/2085] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:30 2022] PUT /identity/v3/projects/b9ebb7f483a84ea5beab691ce0144060/users/9f95b306df78477d9521d5304195ce69/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:30.179401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b7f74-7de5-4edc-8af2-8eecb6b49364 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.179961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b7f74-7de5-4edc-8af2-8eecb6b49364 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.181035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b7f74-7de5-4edc-8af2-8eecb6b49364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.184540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1038/2086] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:30 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:30.185608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505c723-54f3-4b61-9435-f18e0ad82f58 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.215171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-839b7f74-7de5-4edc-8af2-8eecb6b49364 None None] MFA Rules not processed for user `9f95b306df78477d9521d5304195ce69`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:30.216905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-839b7f74-7de5-4edc-8af2-8eecb6b49364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:30.258338 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505c723-54f3-4b61-9435-f18e0ad82f58 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.270226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ac12d-668b-4ef7-9a38-2706e7d3f192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.271811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ac12d-668b-4ef7-9a38-2706e7d3f192 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.272224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ac12d-668b-4ef7-9a38-2706e7d3f192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.272497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ac12d-668b-4ef7-9a38-2706e7d3f192 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.273448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ac12d-668b-4ef7-9a38-2706e7d3f192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.276491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ac12d-668b-4ef7-9a38-2706e7d3f192 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.309776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1049/2087] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:30.316598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b7f74-7de5-4edc-8af2-8eecb6b49364 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.322730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-839b7f74-7de5-4edc-8af2-8eecb6b49364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:30.330625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1039/2088] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:30 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:30.332151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ac12d-668b-4ef7-9a38-2706e7d3f192 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.354117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b7f74-7de5-4edc-8af2-8eecb6b49364 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.362965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ac12d-668b-4ef7-9a38-2706e7d3f192 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.367472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.368411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.368800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.369085 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] PATH_INFO: `/v3/projects/84489b2c376245489f0af5b153e2c860` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.370484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84489b2c376245489f0af5b153e2c860)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.372464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.376308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25fd47-3431-407c-b6b0-0f8e595a23ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.377104 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25fd47-3431-407c-b6b0-0f8e595a23ac admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.377454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25fd47-3431-407c-b6b0-0f8e595a23ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.377810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25fd47-3431-407c-b6b0-0f8e595a23ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.378825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25fd47-3431-407c-b6b0-0f8e595a23ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.381578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25fd47-3431-407c-b6b0-0f8e595a23ac admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.418036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1040/2089] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:30 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:30.419469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25fd47-3431-407c-b6b0-0f8e595a23ac admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.442743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:30.477941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84489b2c376245489f0af5b153e2c860', 'request_id': 'req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:30.487481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84489b2c376245489f0af5b153e2c860', 'request_id': 'req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:30.501344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84489b2c376245489f0af5b153e2c860', 'request_id': 'req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:30.521798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] The token cache is being invalidate because project 84489b2c376245489f0af5b153e2c860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:30.522094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:30.523592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:30.526906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1050/2090] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:30 2022] DELETE /identity/v3/projects/84489b2c376245489f0af5b153e2c860 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:30.527565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.527895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25fd47-3431-407c-b6b0-0f8e595a23ac admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.531221 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:30.587853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:30.600445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097971f-5c59-4a98-a545-0cd792a70c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.601316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097971f-5c59-4a98-a545-0cd792a70c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.602430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097971f-5c59-4a98-a545-0cd792a70c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.602713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097971f-5c59-4a98-a545-0cd792a70c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.603545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097971f-5c59-4a98-a545-0cd792a70c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.605585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097971f-5c59-4a98-a545-0cd792a70c75 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.611654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c7d0a-3fd1-4a88-9bda-05fc6936cd9c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.616508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bc3cd-0705-4b55-bc04-989a77b35db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.620096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1041/2091] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:30.620272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bc3cd-0705-4b55-bc04-989a77b35db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.620272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bc3cd-0705-4b55-bc04-989a77b35db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.620272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bc3cd-0705-4b55-bc04-989a77b35db1 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.620538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097971f-5c59-4a98-a545-0cd792a70c75 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.620623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bc3cd-0705-4b55-bc04-989a77b35db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.620705 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a097971f-5c59-4a98-a545-0cd792a70c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:30.620908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bc3cd-0705-4b55-bc04-989a77b35db1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.632088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097971f-5c59-4a98-a545-0cd792a70c75 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.636918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.637825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.638194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.638495 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] PATH_INFO: `/v3/users/d6b44d46dcc24ad78012de8ebe9ce3f1` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.639812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b44d46dcc24ad78012de8ebe9ce3f1)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.641928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.649393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1051/2092] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:30 2022] POST /identity/v3/users => generated 295 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:30.650246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bc3cd-0705-4b55-bc04-989a77b35db1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.652192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6c0bc3cd-0705-4b55-bc04-989a77b35db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:30.707548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b44d46dcc24ad78012de8ebe9ce3f1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:30.708268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b44d46dcc24ad78012de8ebe9ce3f1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:30.715889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b44d46dcc24ad78012de8ebe9ce3f1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:30.716607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b44d46dcc24ad78012de8ebe9ce3f1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:30.717472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bc3cd-0705-4b55-bc04-989a77b35db1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.721949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505036ef-8edc-4587-82fa-adafea3a8b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.722616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505036ef-8edc-4587-82fa-adafea3a8b89 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.722901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505036ef-8edc-4587-82fa-adafea3a8b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.723181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505036ef-8edc-4587-82fa-adafea3a8b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.725077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-505036ef-8edc-4587-82fa-adafea3a8b89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:30.732157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b44d46dcc24ad78012de8ebe9ce3f1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:30.754118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1042/2093] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:30 2022] DELETE /identity/v3/users/d6b44d46dcc24ad78012de8ebe9ce3f1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:30.757174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.757914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:30.812430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:30.817596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505036ef-8edc-4587-82fa-adafea3a8b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.820263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505036ef-8edc-4587-82fa-adafea3a8b89 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.844382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1052/2094] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:30.845932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505036ef-8edc-4587-82fa-adafea3a8b89 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.848511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-505036ef-8edc-4587-82fa-adafea3a8b89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:30.863198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dc9ba-4319-45a4-93ad-82d844823632 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.869173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0591622-afbd-40d1-9b39-fc790fb22c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.869879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0591622-afbd-40d1-9b39-fc790fb22c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.870156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0591622-afbd-40d1-9b39-fc790fb22c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.870430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0591622-afbd-40d1-9b39-fc790fb22c7e admin admin] PATH_INFO: `/v3/projects/419613ac24fb4a8bb0b190c5c101f532/users/834df9c33a814dc69af5f9e7eaee930e/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.872651 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505036ef-8edc-4587-82fa-adafea3a8b89 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:30.880920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc14d5-3904-4a32-904e-ca07889be047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:30.882197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc14d5-3904-4a32-904e-ca07889be047 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:30.882630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc14d5-3904-4a32-904e-ca07889be047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:30.883114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc14d5-3904-4a32-904e-ca07889be047 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:30.884838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc14d5-3904-4a32-904e-ca07889be047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.887462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc14d5-3904-4a32-904e-ca07889be047 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.888802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0591622-afbd-40d1-9b39-fc790fb22c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=419613ac24fb4a8bb0b190c5c101f532, user_id=834df9c33a814dc69af5f9e7eaee930e, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:30.893601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0591622-afbd-40d1-9b39-fc790fb22c7e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:30.907932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1053/2095] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:30.912063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc14d5-3904-4a32-904e-ca07889be047 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.912868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1043/2096] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:30 2022] PUT /identity/v3/projects/419613ac24fb4a8bb0b190c5c101f532/users/834df9c33a814dc69af5f9e7eaee930e/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:30.917453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0591622-afbd-40d1-9b39-fc790fb22c7e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:30.925903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc14d5-3904-4a32-904e-ca07889be047 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:31.057782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7cd922-d69a-4b99-8f82-50c8d588eeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:31.061192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7cd922-d69a-4b99-8f82-50c8d588eeec admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.061669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7cd922-d69a-4b99-8f82-50c8d588eeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.062671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0591622-afbd-40d1-9b39-fc790fb22c7e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:31.062986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7cd922-d69a-4b99-8f82-50c8d588eeec admin admin] PATH_INFO: `/v3/projects/68ad6fc409374a77a5ae0602f2213168/users/3bda5a3396e54d07beae222af0183c9b/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.070940 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39986194-0eb8-4285-948a-172fa8085be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:31.071961 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39986194-0eb8-4285-948a-172fa8085be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.072601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39986194-0eb8-4285-948a-172fa8085be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.073072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39986194-0eb8-4285-948a-172fa8085be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.074133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39986194-0eb8-4285-948a-172fa8085be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:31.077063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39986194-0eb8-4285-948a-172fa8085be2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:31.089888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7cd922-d69a-4b99-8f82-50c8d588eeec admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ad6fc409374a77a5ae0602f2213168, user_id=3bda5a3396e54d07beae222af0183c9b, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:31.094368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1044/2097] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:31.095757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39986194-0eb8-4285-948a-172fa8085be2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:31.103623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7cd922-d69a-4b99-8f82-50c8d588eeec admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:31.108802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39986194-0eb8-4285-948a-172fa8085be2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:31.115797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1cfb4-cec5-47be-8dcf-49e6c7862cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:31.116451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:31.116988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1cfb4-cec5-47be-8dcf-49e6c7862cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.117406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1cfb4-cec5-47be-8dcf-49e6c7862cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.117805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1cfb4-cec5-47be-8dcf-49e6c7862cc9 admin admin] PATH_INFO: `/v3/projects/419613ac24fb4a8bb0b190c5c101f532/users/834df9c33a814dc69af5f9e7eaee930e/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.121338 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1cfb4-cec5-47be-8dcf-49e6c7862cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=419613ac24fb4a8bb0b190c5c101f532, user_id=834df9c33a814dc69af5f9e7eaee930e, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:31.124964 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1cfb4-cec5-47be-8dcf-49e6c7862cc9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:31.141119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1054/2098] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:30 2022] PUT /identity/v3/projects/68ad6fc409374a77a5ae0602f2213168/users/3bda5a3396e54d07beae222af0183c9b/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:31.142072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7cd922-d69a-4b99-8f82-50c8d588eeec admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:31.144326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1045/2099] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:31 2022] PUT /identity/v3/projects/419613ac24fb4a8bb0b190c5c101f532/users/834df9c33a814dc69af5f9e7eaee930e/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:31.159507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a65584-6ff1-4cf8-90b1-0adcc8e9c3d2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.159507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a65584-6ff1-4cf8-90b1-0adcc8e9c3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.159507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a65584-6ff1-4cf8-90b1-0adcc8e9c3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.192839 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b8a65584-6ff1-4cf8-90b1-0adcc8e9c3d2 None None] MFA Rules not processed for user `834df9c33a814dc69af5f9e7eaee930e`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:31.194565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b8a65584-6ff1-4cf8-90b1-0adcc8e9c3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:31.241806 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7cd922-d69a-4b99-8f82-50c8d588eeec admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:31.305086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1046/2100] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:31.321164 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a65584-6ff1-4cf8-90b1-0adcc8e9c3d2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:31.322753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b8a65584-6ff1-4cf8-90b1-0adcc8e9c3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:31.338091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed9345-f4dc-479e-b308-d118a0a9ecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:31.339089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9345-f4dc-479e-b308-d118a0a9ecd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.339458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9345-f4dc-479e-b308-d118a0a9ecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.340308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9345-f4dc-479e-b308-d118a0a9ecd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.341632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed9345-f4dc-479e-b308-d118a0a9ecd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:31.343434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed9345-f4dc-479e-b308-d118a0a9ecd6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:31.354526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:31.361111 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1055/2101] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:31.362301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed9345-f4dc-479e-b308-d118a0a9ecd6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:31.373780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed9345-f4dc-479e-b308-d118a0a9ecd6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:31.378941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7fe69-6dda-4b2e-83fd-cd742eb36d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:31.379973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7fe69-6dda-4b2e-83fd-cd742eb36d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.380391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7fe69-6dda-4b2e-83fd-cd742eb36d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.380761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7fe69-6dda-4b2e-83fd-cd742eb36d81 admin admin] PATH_INFO: `/v3/projects/68ad6fc409374a77a5ae0602f2213168/users/3bda5a3396e54d07beae222af0183c9b/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.383110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7fe69-6dda-4b2e-83fd-cd742eb36d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ad6fc409374a77a5ae0602f2213168, user_id=3bda5a3396e54d07beae222af0183c9b, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:31.386607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7fe69-6dda-4b2e-83fd-cd742eb36d81 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:31.397934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a65584-6ff1-4cf8-90b1-0adcc8e9c3d2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:31.401583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1056/2102] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:31 2022] PUT /identity/v3/projects/68ad6fc409374a77a5ae0602f2213168/users/3bda5a3396e54d07beae222af0183c9b/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:31.413167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf027277-0247-4465-bf77-c496fa879a93 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.413654 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf027277-0247-4465-bf77-c496fa879a93 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.415169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf027277-0247-4465-bf77-c496fa879a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.483510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cf027277-0247-4465-bf77-c496fa879a93 None None] MFA Rules not processed for user `3bda5a3396e54d07beae222af0183c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:31.484451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cf027277-0247-4465-bf77-c496fa879a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:31.506531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502e74f-407a-4beb-802b-64b95d218536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:31.507510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502e74f-407a-4beb-802b-64b95d218536 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.508233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502e74f-407a-4beb-802b-64b95d218536 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.508704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502e74f-407a-4beb-802b-64b95d218536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.631329 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502e74f-407a-4beb-802b-64b95d218536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:31.633045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502e74f-407a-4beb-802b-64b95d218536 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:31.657591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1047/2103] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:31.658653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502e74f-407a-4beb-802b-64b95d218536 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:31.660278 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6502e74f-407a-4beb-802b-64b95d218536 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:31.685810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1057/2104] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:31.747800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf027277-0247-4465-bf77-c496fa879a93 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:31.761299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf027277-0247-4465-bf77-c496fa879a93 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:31.761481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502e74f-407a-4beb-802b-64b95d218536 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:31.768480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:31.769339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0313830-f725-4de0-9251-81b5b4192370 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:31.770014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.770325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0313830-f725-4de0-9251-81b5b4192370 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.770492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.770900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0313830-f725-4de0-9251-81b5b4192370 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.771106 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] PATH_INFO: `/v3/projects/d8d54e6d6142497981b69b989ca6d131` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.771465 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0313830-f725-4de0-9251-81b5b4192370 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.774952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d54e6d6142497981b69b989ca6d131)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:31.777175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:31.777291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f0313830-f725-4de0-9251-81b5b4192370 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:31.802627 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:31.855750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:31.878105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d54e6d6142497981b69b989ca6d131', 'request_id': 'req-4a640e53-dfbe-4375-b478-168261d2c3f4'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:31.886277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d54e6d6142497981b69b989ca6d131', 'request_id': 'req-4a640e53-dfbe-4375-b478-168261d2c3f4'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:31.894720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0313830-f725-4de0-9251-81b5b4192370 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:31.896423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0313830-f725-4de0-9251-81b5b4192370 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:31.896709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d54e6d6142497981b69b989ca6d131', 'request_id': 'req-4a640e53-dfbe-4375-b478-168261d2c3f4'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:31.907770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1058/2105] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:31.909131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0313830-f725-4de0-9251-81b5b4192370 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:31.910806 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f0313830-f725-4de0-9251-81b5b4192370 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:31.912832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] The token cache is being invalidate because project d8d54e6d6142497981b69b989ca6d131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:31.913194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:31.914559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a640e53-dfbe-4375-b478-168261d2c3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:31.920317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1048/2106] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:31 2022] DELETE /identity/v3/projects/d8d54e6d6142497981b69b989ca6d131 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:31.921855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210326e-0397-4e92-b30f-924d588d6691 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.922145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210326e-0397-4e92-b30f-924d588d6691 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.922441 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210326e-0397-4e92-b30f-924d588d6691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:31.923022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0313830-f725-4de0-9251-81b5b4192370 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:31.962820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f210326e-0397-4e92-b30f-924d588d6691 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:31.964649 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f210326e-0397-4e92-b30f-924d588d6691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:31.997130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49704013-2842-4f60-ac1e-6b153dcf8faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:31.997650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49704013-2842-4f60-ac1e-6b153dcf8faf service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:31.997930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49704013-2842-4f60-ac1e-6b153dcf8faf service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:31.998200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49704013-2842-4f60-ac1e-6b153dcf8faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.000188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-49704013-2842-4f60-ac1e-6b153dcf8faf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:32.064762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1049/2107] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:32.074004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03556877-7e98-45eb-b47c-656a47f59d94 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:32.076521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03556877-7e98-45eb-b47c-656a47f59d94 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:32.076831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03556877-7e98-45eb-b47c-656a47f59d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.104659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49704013-2842-4f60-ac1e-6b153dcf8faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:32.105709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-03556877-7e98-45eb-b47c-656a47f59d94 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:32.107197 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-03556877-7e98-45eb-b47c-656a47f59d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:32.108050 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49704013-2842-4f60-ac1e-6b153dcf8faf service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:32.117170 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1050/2108] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:32.121008 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03556877-7e98-45eb-b47c-656a47f59d94 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:32.123390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1059/2109] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:32.134836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03556877-7e98-45eb-b47c-656a47f59d94 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:32.142305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8258e12-0efd-4aad-94f2-f9c22b1aecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:32.143861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8258e12-0efd-4aad-94f2-f9c22b1aecce admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:32.144596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8258e12-0efd-4aad-94f2-f9c22b1aecce admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:32.145046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8258e12-0efd-4aad-94f2-f9c22b1aecce admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.145957 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8258e12-0efd-4aad-94f2-f9c22b1aecce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:32.146411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8258e12-0efd-4aad-94f2-f9c22b1aecce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:32.148391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8258e12-0efd-4aad-94f2-f9c22b1aecce admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:32.169010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1051/2110] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:58:32.173571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49704013-2842-4f60-ac1e-6b153dcf8faf service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:32.189456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49704013-2842-4f60-ac1e-6b153dcf8faf service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:32.196771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeecf1a-90a0-47a9-8430-19999554d544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:32.198079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeecf1a-90a0-47a9-8430-19999554d544 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:32.198449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeecf1a-90a0-47a9-8430-19999554d544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:32.198828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeecf1a-90a0-47a9-8430-19999554d544 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.199841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeecf1a-90a0-47a9-8430-19999554d544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:32.202006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeecf1a-90a0-47a9-8430-19999554d544 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:32.238143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1060/2111] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:32 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:32.241991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8258e12-0efd-4aad-94f2-f9c22b1aecce admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:32.253233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8258e12-0efd-4aad-94f2-f9c22b1aecce admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:32.325755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeecf1a-90a0-47a9-8430-19999554d544 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:32.327302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cbeecf1a-90a0-47a9-8430-19999554d544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:32.365520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b15055-98b0-4bcc-8973-7407a3a54349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:32.366254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b15055-98b0-4bcc-8973-7407a3a54349 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:32.367204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b15055-98b0-4bcc-8973-7407a3a54349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:32.368431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b15055-98b0-4bcc-8973-7407a3a54349 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.370672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b15055-98b0-4bcc-8973-7407a3a54349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:32.376182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b15055-98b0-4bcc-8973-7407a3a54349 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:32.419317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1052/2112] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:32 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:32.423841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b15055-98b0-4bcc-8973-7407a3a54349 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:32.435425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:32.435860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b15055-98b0-4bcc-8973-7407a3a54349 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:32.441717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e0396-3a0d-4a22-9698-5034d14c665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:32.442701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e0396-3a0d-4a22-9698-5034d14c665f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:32.443006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e0396-3a0d-4a22-9698-5034d14c665f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:32.443230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e0396-3a0d-4a22-9698-5034d14c665f admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.444254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e0396-3a0d-4a22-9698-5034d14c665f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:32.445940 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e0396-3a0d-4a22-9698-5034d14c665f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:32.463034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1053/2113] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:58:32.464408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeecf1a-90a0-47a9-8430-19999554d544 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:32.467084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e0396-3a0d-4a22-9698-5034d14c665f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:32.474576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe266-2ffb-494e-b73e-5ef04046440d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:32.476299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe266-2ffb-494e-b73e-5ef04046440d service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:32.476584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe266-2ffb-494e-b73e-5ef04046440d service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:32.476857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe266-2ffb-494e-b73e-5ef04046440d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.482363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d7dbe266-2ffb-494e-b73e-5ef04046440d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:32.492029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e0396-3a0d-4a22-9698-5034d14c665f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:32.507319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bef2f2-2ace-4191-a0f5-4e5c55481886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:32.510446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bef2f2-2ace-4191-a0f5-4e5c55481886 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:32.510747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bef2f2-2ace-4191-a0f5-4e5c55481886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:32.513051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bef2f2-2ace-4191-a0f5-4e5c55481886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.515359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bef2f2-2ace-4191-a0f5-4e5c55481886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:32.520252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bef2f2-2ace-4191-a0f5-4e5c55481886 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:32.546626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1054/2114] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:32.552183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bef2f2-2ace-4191-a0f5-4e5c55481886 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:32.572832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bef2f2-2ace-4191-a0f5-4e5c55481886 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:32.578993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04e393-4012-4ac1-ae7d-fac5b0b73b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:32.580053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04e393-4012-4ac1-ae7d-fac5b0b73b07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:32.580396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04e393-4012-4ac1-ae7d-fac5b0b73b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:32.580766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04e393-4012-4ac1-ae7d-fac5b0b73b07 admin admin] PATH_INFO: `/v3/projects/3797efb640e743978281ecc69d921b8d/users/0a1eb138d4014c09a1c4421c469620a2/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.583713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:32.600222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbe266-2ffb-494e-b73e-5ef04046440d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:32.602683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbe266-2ffb-494e-b73e-5ef04046440d service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:32.604212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04e393-4012-4ac1-ae7d-fac5b0b73b07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3797efb640e743978281ecc69d921b8d, user_id=0a1eb138d4014c09a1c4421c469620a2, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:32.607953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04e393-4012-4ac1-ae7d-fac5b0b73b07 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:32.627891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1061/2115] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:32.631224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1055/2116] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:32 2022] PUT /identity/v3/projects/3797efb640e743978281ecc69d921b8d/users/0a1eb138d4014c09a1c4421c469620a2/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:32.643309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6826-4e0c-477e-8440-1101470d8ca7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:32.643603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6826-4e0c-477e-8440-1101470d8ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:32.643882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6826-4e0c-477e-8440-1101470d8ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.678410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-153c6826-4e0c-477e-8440-1101470d8ca7 None None] MFA Rules not processed for user `0a1eb138d4014c09a1c4421c469620a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:32.679091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-153c6826-4e0c-477e-8440-1101470d8ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:32.770587 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1062/2117] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:32.775010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04e393-4012-4ac1-ae7d-fac5b0b73b07 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:32.926640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04e393-4012-4ac1-ae7d-fac5b0b73b07 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:32.933485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3a305-43da-4453-94e2-31170ca441f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:32.939084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3a305-43da-4453-94e2-31170ca441f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:32.941057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3a305-43da-4453-94e2-31170ca441f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:32.941498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3a305-43da-4453-94e2-31170ca441f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:32.942341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3a305-43da-4453-94e2-31170ca441f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:32.948933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3a305-43da-4453-94e2-31170ca441f4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:32.999381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1056/2118] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:32 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:33.003602 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6826-4e0c-477e-8440-1101470d8ca7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:33.016757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6826-4e0c-477e-8440-1101470d8ca7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:33.082074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9de7b-169c-48c4-8488-2e258fedb3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:33.082770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9de7b-169c-48c4-8488-2e258fedb3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:33.083703 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9de7b-169c-48c4-8488-2e258fedb3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:33.083703 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9de7b-169c-48c4-8488-2e258fedb3c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:33.084021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9de7b-169c-48c4-8488-2e258fedb3c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:33.086075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9de7b-169c-48c4-8488-2e258fedb3c9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:33.124038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1063/2119] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:33 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:33.125374 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3a305-43da-4453-94e2-31170ca441f4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:33.136269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3a305-43da-4453-94e2-31170ca441f4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:33.141738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214320a-8cc2-4d44-8033-acd53d39c4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:33.142505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214320a-8cc2-4d44-8033-acd53d39c4ab admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:33.142987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214320a-8cc2-4d44-8033-acd53d39c4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:33.143455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214320a-8cc2-4d44-8033-acd53d39c4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:33.144294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214320a-8cc2-4d44-8033-acd53d39c4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:33.145977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214320a-8cc2-4d44-8033-acd53d39c4ab admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:33.160690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1057/2120] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:33.162110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9de7b-169c-48c4-8488-2e258fedb3c9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:33.176018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9de7b-169c-48c4-8488-2e258fedb3c9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:33.181894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18d215-8c04-46cb-8ac6-21cf626aa9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:33.182892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18d215-8c04-46cb-8ac6-21cf626aa9f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:33.183224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18d215-8c04-46cb-8ac6-21cf626aa9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:33.183501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18d215-8c04-46cb-8ac6-21cf626aa9f2 admin admin] PATH_INFO: `/v3/projects/3a672b6cac1e4751bc348e15ce708ff6/users/efb814fcaa4449769b09c4d45aaab613/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:33.197016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:33.205346 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18d215-8c04-46cb-8ac6-21cf626aa9f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a672b6cac1e4751bc348e15ce708ff6, user_id=efb814fcaa4449769b09c4d45aaab613, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:33.209109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18d215-8c04-46cb-8ac6-21cf626aa9f2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:33.246930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1064/2121] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:33 2022] PUT /identity/v3/projects/3a672b6cac1e4751bc348e15ce708ff6/users/efb814fcaa4449769b09c4d45aaab613/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:33.251151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214320a-8cc2-4d44-8033-acd53d39c4ab admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:33.267007 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214320a-8cc2-4d44-8033-acd53d39c4ab admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:33.356057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe35c9-0968-4e74-88f1-af762b3854ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:33.356941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe35c9-0968-4e74-88f1-af762b3854ee admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:33.357230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe35c9-0968-4e74-88f1-af762b3854ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:33.357508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe35c9-0968-4e74-88f1-af762b3854ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:33.358260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe35c9-0968-4e74-88f1-af762b3854ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:33.359890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe35c9-0968-4e74-88f1-af762b3854ee admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:33.376709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1058/2122] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:33.378075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18d215-8c04-46cb-8ac6-21cf626aa9f2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:33.391606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18d215-8c04-46cb-8ac6-21cf626aa9f2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:33.397649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaea83-a852-47d6-9423-51d2e30c69ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:33.398809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaea83-a852-47d6-9423-51d2e30c69ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:33.400938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaea83-a852-47d6-9423-51d2e30c69ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:33.401303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaea83-a852-47d6-9423-51d2e30c69ec admin admin] PATH_INFO: `/v3/projects/3a672b6cac1e4751bc348e15ce708ff6/users/efb814fcaa4449769b09c4d45aaab613/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:33.403492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaea83-a852-47d6-9423-51d2e30c69ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a672b6cac1e4751bc348e15ce708ff6, user_id=efb814fcaa4449769b09c4d45aaab613, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:33.406694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaea83-a852-47d6-9423-51d2e30c69ec admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:33.428900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1065/2123] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:33 2022] PUT /identity/v3/projects/3a672b6cac1e4751bc348e15ce708ff6/users/efb814fcaa4449769b09c4d45aaab613/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:33.435505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745777d-6f47-42c0-b46f-89748960ead0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:33.435694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745777d-6f47-42c0-b46f-89748960ead0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:33.436011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745777d-6f47-42c0-b46f-89748960ead0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:33.503622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5745777d-6f47-42c0-b46f-89748960ead0 None None] MFA Rules not processed for user `efb814fcaa4449769b09c4d45aaab613`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:33.506594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5745777d-6f47-42c0-b46f-89748960ead0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:33.561771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:33.622168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1059/2124] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:33.630037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaea83-a852-47d6-9423-51d2e30c69ec admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:33.631832 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-99aaea83-a852-47d6-9423-51d2e30c69ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:33.703768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaea83-a852-47d6-9423-51d2e30c69ec admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:33.707921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92df76a-d383-49dd-bbbd-7c0eecb48b19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:33.708767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92df76a-d383-49dd-bbbd-7c0eecb48b19 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:33.709052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92df76a-d383-49dd-bbbd-7c0eecb48b19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:33.709326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92df76a-d383-49dd-bbbd-7c0eecb48b19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:33.720600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92df76a-d383-49dd-bbbd-7c0eecb48b19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:33.722828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92df76a-d383-49dd-bbbd-7c0eecb48b19 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:33.734083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1066/2125] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:33.904948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745777d-6f47-42c0-b46f-89748960ead0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:33.915926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745777d-6f47-42c0-b46f-89748960ead0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:33.921006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea478fe-8fb8-49e5-89e8-c8ff0aee80d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:33.921682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea478fe-8fb8-49e5-89e8-c8ff0aee80d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:33.921965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea478fe-8fb8-49e5-89e8-c8ff0aee80d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:33.922241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea478fe-8fb8-49e5-89e8-c8ff0aee80d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:33.983181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea478fe-8fb8-49e5-89e8-c8ff0aee80d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:33.984876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea478fe-8fb8-49e5-89e8-c8ff0aee80d0 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:33.997165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1060/2126] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:34.025839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92df76a-d383-49dd-bbbd-7c0eecb48b19 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:34.036197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92df76a-d383-49dd-bbbd-7c0eecb48b19 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:34.039871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e2929-9adb-49f2-9fcc-7ebd094abdd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:34.040653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e2929-9adb-49f2-9fcc-7ebd094abdd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:34.040996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e2929-9adb-49f2-9fcc-7ebd094abdd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:34.041336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e2929-9adb-49f2-9fcc-7ebd094abdd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:34.043166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-300e2929-9adb-49f2-9fcc-7ebd094abdd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:34.109213 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e2929-9adb-49f2-9fcc-7ebd094abdd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:34.110829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e2929-9adb-49f2-9fcc-7ebd094abdd4 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:34.136445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1067/2127] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:35.242002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea478fe-8fb8-49e5-89e8-c8ff0aee80d0 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:35.253770 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea478fe-8fb8-49e5-89e8-c8ff0aee80d0 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:35.260137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bc035-bae7-4b35-8780-e931a557df85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:35.262212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bc035-bae7-4b35-8780-e931a557df85 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:35.262212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bc035-bae7-4b35-8780-e931a557df85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:35.262212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bc035-bae7-4b35-8780-e931a557df85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:35.266379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-489bc035-bae7-4b35-8780-e931a557df85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:35.386953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bc035-bae7-4b35-8780-e931a557df85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:35.388680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bc035-bae7-4b35-8780-e931a557df85 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:35.400473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1061/2128] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:35.512637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e2929-9adb-49f2-9fcc-7ebd094abdd4 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:35.593182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e2929-9adb-49f2-9fcc-7ebd094abdd4 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:35.598686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:35.599790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:35.599904 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:35.600427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] PATH_INFO: `/v3/users/0a1eb138d4014c09a1c4421c469620a2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:35.602108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1eb138d4014c09a1c4421c469620a2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:35.604392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:35.627413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:35.679077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1eb138d4014c09a1c4421c469620a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:35.687392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1eb138d4014c09a1c4421c469620a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:35.688165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1eb138d4014c09a1c4421c469620a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:35.700308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1eb138d4014c09a1c4421c469620a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:35.719339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1eb138d4014c09a1c4421c469620a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:35.728175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1068/2129] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:35 2022] DELETE /identity/v3/users/0a1eb138d4014c09a1c4421c469620a2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:35.733399 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8f586-8a41-4ef4-8380-466a92dae955 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:35.733711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8f586-8a41-4ef4-8380-466a92dae955 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:35.734168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8f586-8a41-4ef4-8380-466a92dae955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:35.759577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a6a8f586-8a41-4ef4-8380-466a92dae955 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:35.761392 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a6a8f586-8a41-4ef4-8380-466a92dae955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:35.860921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1062/2130] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:35.872512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:35.874105 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:35.940979 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31036a8-73e7-4bd7-8d20-775a42ec8c78 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:35.945256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b302d9-8111-48fb-9dd6-f022aed959a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:35.946256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b302d9-8111-48fb-9dd6-f022aed959a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:35.946977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b302d9-8111-48fb-9dd6-f022aed959a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:35.947626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b302d9-8111-48fb-9dd6-f022aed959a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:35.959454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b302d9-8111-48fb-9dd6-f022aed959a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:35.961226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b302d9-8111-48fb-9dd6-f022aed959a3 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:35.972054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1069/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:36.354265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8f586-8a41-4ef4-8380-466a92dae955 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:36.356766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a6a8f586-8a41-4ef4-8380-466a92dae955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:36.388590 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8f586-8a41-4ef4-8380-466a92dae955 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:36.400030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:36.400989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:36.401273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:36.401550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] PATH_INFO: `/v3/projects/3797efb640e743978281ecc69d921b8d` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:36.402957 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3797efb640e743978281ecc69d921b8d)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:36.404960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:36.415140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:36.482743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3797efb640e743978281ecc69d921b8d', 'request_id': 'req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:36.492328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3797efb640e743978281ecc69d921b8d', 'request_id': 'req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:36.503703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3797efb640e743978281ecc69d921b8d', 'request_id': 'req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:36.527678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] The token cache is being invalidate because project 3797efb640e743978281ecc69d921b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:36.528255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:36.531838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:36.531838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1063/2132] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:36 2022] DELETE /identity/v3/projects/3797efb640e743978281ecc69d921b8d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:36.537109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b302d9-8111-48fb-9dd6-f022aed959a3 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:36.552009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b302d9-8111-48fb-9dd6-f022aed959a3 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:36.638469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:36.640224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:36.640596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:36.641638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] PATH_INFO: `/v3/users/efb814fcaa4449769b09c4d45aaab613` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:36.643854 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=efb814fcaa4449769b09c4d45aaab613)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:36.646705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:36.692847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:36.699600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efb814fcaa4449769b09c4d45aaab613'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:36.707437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efb814fcaa4449769b09c4d45aaab613'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:36.708133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efb814fcaa4449769b09c4d45aaab613'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:36.716631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efb814fcaa4449769b09c4d45aaab613'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:36.729792 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efb814fcaa4449769b09c4d45aaab613'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:36.736672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1070/2133] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:36 2022] DELETE /identity/v3/users/efb814fcaa4449769b09c4d45aaab613 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:36.996721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:36.998151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:37.013679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c8e4-d552-4faa-be99-a2d2857a4bc7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.074314 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.075221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.075506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.075784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] PATH_INFO: `/v3/users/96025df19bb64985908fc5187af44888` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.077390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96025df19bb64985908fc5187af44888)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.079098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.135593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.136525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96025df19bb64985908fc5187af44888'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.137273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:37.137403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96025df19bb64985908fc5187af44888'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.146304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96025df19bb64985908fc5187af44888'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.147628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96025df19bb64985908fc5187af44888'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.151348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25b745-686a-4d3f-8d8b-2a5e5376fe3f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.159501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.160432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.160711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.160983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] PATH_INFO: `/v3/projects/3a672b6cac1e4751bc348e15ce708ff6` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.162279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a672b6cac1e4751bc348e15ce708ff6)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.164262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.172711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e400fa71-a591-4b28-9cb0-b0570bae63f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96025df19bb64985908fc5187af44888'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.193763 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1064/2134] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:36 2022] DELETE /identity/v3/users/96025df19bb64985908fc5187af44888 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:37.198027 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92587-5568-407d-b247-c63ef0eaf554 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.198315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92587-5568-407d-b247-c63ef0eaf554 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.198639 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92587-5568-407d-b247-c63ef0eaf554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.215946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-47a92587-5568-407d-b247-c63ef0eaf554 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:37.217181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a672b6cac1e4751bc348e15ce708ff6', 'request_id': 'req-ad8c5886-88be-4c7b-9da8-2e363a446d18'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.217310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-47a92587-5568-407d-b247-c63ef0eaf554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:37.224189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a672b6cac1e4751bc348e15ce708ff6', 'request_id': 'req-ad8c5886-88be-4c7b-9da8-2e363a446d18'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.225542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:37.232976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a672b6cac1e4751bc348e15ce708ff6', 'request_id': 'req-ad8c5886-88be-4c7b-9da8-2e363a446d18'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.243934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] The token cache is being invalidate because project 3a672b6cac1e4751bc348e15ce708ff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:37.244247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.244889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad8c5886-88be-4c7b-9da8-2e363a446d18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.250706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1071/2135] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:37 2022] DELETE /identity/v3/projects/3a672b6cac1e4751bc348e15ce708ff6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:37.252205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f8b8-427d-4456-a337-198bff8823d9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.252513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f8b8-427d-4456-a337-198bff8823d9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.252789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f8b8-427d-4456-a337-198bff8823d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.271278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4c33f8b8-427d-4456-a337-198bff8823d9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:37.272531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4c33f8b8-427d-4456-a337-198bff8823d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:37.284806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1065/2136] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:37.295183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a92587-5568-407d-b247-c63ef0eaf554 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.296635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-47a92587-5568-407d-b247-c63ef0eaf554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:37.333486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1072/2137] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:37.341254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda7442-6a35-454e-adda-11b0d7c1b475 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.341544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda7442-6a35-454e-adda-11b0d7c1b475 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.341821 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda7442-6a35-454e-adda-11b0d7c1b475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.354811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a92587-5568-407d-b247-c63ef0eaf554 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.358481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d4e26-71e6-4aec-8b65-4b83e14f9cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.359131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d4e26-71e6-4aec-8b65-4b83e14f9cbb service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.359395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d4e26-71e6-4aec-8b65-4b83e14f9cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.359509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4bda7442-6a35-454e-adda-11b0d7c1b475 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:37.359697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d4e26-71e6-4aec-8b65-4b83e14f9cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.360713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4bda7442-6a35-454e-adda-11b0d7c1b475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:37.369173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d4e26-71e6-4aec-8b65-4b83e14f9cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.369599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1073/2138] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:37.370708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d4e26-71e6-4aec-8b65-4b83e14f9cbb service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.373803 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda7442-6a35-454e-adda-11b0d7c1b475 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.381050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1066/2139] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:37.383208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda7442-6a35-454e-adda-11b0d7c1b475 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.387651 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb67832-5fa5-4106-8fc4-25b3f70aa1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.388356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb67832-5fa5-4106-8fc4-25b3f70aa1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.388637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb67832-5fa5-4106-8fc4-25b3f70aa1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.388910 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb67832-5fa5-4106-8fc4-25b3f70aa1c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.389633 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb67832-5fa5-4106-8fc4-25b3f70aa1c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:37.389935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb67832-5fa5-4106-8fc4-25b3f70aa1c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.391537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb67832-5fa5-4106-8fc4-25b3f70aa1c9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.403428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1074/2140] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:58:37.407951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d4e26-71e6-4aec-8b65-4b83e14f9cbb service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.418087 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d4e26-71e6-4aec-8b65-4b83e14f9cbb service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.423254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c2a1b-bebe-4850-bdc0-643833c289aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.424046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c2a1b-bebe-4850-bdc0-643833c289aa admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.424428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c2a1b-bebe-4850-bdc0-643833c289aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.425035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c2a1b-bebe-4850-bdc0-643833c289aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.425760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c2a1b-bebe-4850-bdc0-643833c289aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.427855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c2a1b-bebe-4850-bdc0-643833c289aa admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.456723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1067/2141] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:37 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:37.460985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb67832-5fa5-4106-8fc4-25b3f70aa1c9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.471263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb67832-5fa5-4106-8fc4-25b3f70aa1c9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.490019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:37.539821 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d708a6-c634-4e60-a11d-54a8b00cd1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.540555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d708a6-c634-4e60-a11d-54a8b00cd1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.540906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d708a6-c634-4e60-a11d-54a8b00cd1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.541091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d708a6-c634-4e60-a11d-54a8b00cd1d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.542313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d708a6-c634-4e60-a11d-54a8b00cd1d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.544368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d708a6-c634-4e60-a11d-54a8b00cd1d3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.568843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1075/2142] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:37 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:58:37.570088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c2a1b-bebe-4850-bdc0-643833c289aa admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.581276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c2a1b-bebe-4850-bdc0-643833c289aa admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.586190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ab4b5-08a7-4101-a05f-d4b8d14a1429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.586859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ab4b5-08a7-4101-a05f-d4b8d14a1429 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.587138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ab4b5-08a7-4101-a05f-d4b8d14a1429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.587412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ab4b5-08a7-4101-a05f-d4b8d14a1429 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.588194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ab4b5-08a7-4101-a05f-d4b8d14a1429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.589797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ab4b5-08a7-4101-a05f-d4b8d14a1429 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.602271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1068/2143] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:37.606840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d708a6-c634-4e60-a11d-54a8b00cd1d3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.617991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d708a6-c634-4e60-a11d-54a8b00cd1d3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.623660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea9dbb-22c8-4469-9449-132689f48823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.624388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9dbb-22c8-4469-9449-132689f48823 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.624674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9dbb-22c8-4469-9449-132689f48823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.624945 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9dbb-22c8-4469-9449-132689f48823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.625675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea9dbb-22c8-4469-9449-132689f48823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.627455 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea9dbb-22c8-4469-9449-132689f48823 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.638220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1076/2144] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:37.642267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ab4b5-08a7-4101-a05f-d4b8d14a1429 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.654016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ab4b5-08a7-4101-a05f-d4b8d14a1429 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.661306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b798a-e58f-4dc9-8b86-5ff30d122187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.662006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea9dbb-22c8-4469-9449-132689f48823 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.662413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b798a-e58f-4dc9-8b86-5ff30d122187 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.662874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b798a-e58f-4dc9-8b86-5ff30d122187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.663142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b798a-e58f-4dc9-8b86-5ff30d122187 admin admin] PATH_INFO: `/v3/projects/55b6d41ed94b4765926d065244f1f2a8/users/87fe34dc828e4169aa5f7b7e876351ff/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.664219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-08ea9dbb-22c8-4469-9449-132689f48823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:37.681773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea9dbb-22c8-4469-9449-132689f48823 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.687033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.688000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.688368 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.688707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] PATH_INFO: `/v3/projects/1a3c0d48f55c4b6cb4a7e5d772cd18a2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.690124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b798a-e58f-4dc9-8b86-5ff30d122187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55b6d41ed94b4765926d065244f1f2a8, user_id=87fe34dc828e4169aa5f7b7e876351ff, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.691307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a3c0d48f55c4b6cb4a7e5d772cd18a2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.693274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.693655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b798a-e58f-4dc9-8b86-5ff30d122187 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.716146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1069/2145] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:37 2022] PUT /identity/v3/projects/55b6d41ed94b4765926d065244f1f2a8/users/87fe34dc828e4169aa5f7b7e876351ff/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:37.728320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0945f5a-b518-4b6a-b5f6-0a3e73c08018 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.728805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0945f5a-b518-4b6a-b5f6-0a3e73c08018 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.729088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0945f5a-b518-4b6a-b5f6-0a3e73c08018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.759692 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a3c0d48f55c4b6cb4a7e5d772cd18a2', 'request_id': 'req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.760697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a0945f5a-b518-4b6a-b5f6-0a3e73c08018 None None] MFA Rules not processed for user `87fe34dc828e4169aa5f7b7e876351ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:37.762392 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a0945f5a-b518-4b6a-b5f6-0a3e73c08018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:37.767405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a3c0d48f55c4b6cb4a7e5d772cd18a2', 'request_id': 'req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.775729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a3c0d48f55c4b6cb4a7e5d772cd18a2', 'request_id': 'req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.786463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] The token cache is being invalidate because project 1a3c0d48f55c4b6cb4a7e5d772cd18a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:37.786753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.787397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.789138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1077/2146] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:37 2022] DELETE /identity/v3/projects/1a3c0d48f55c4b6cb4a7e5d772cd18a2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:37.789909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.791193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:37.813306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:37.837680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1070/2147] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:37.838696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0945f5a-b518-4b6a-b5f6-0a3e73c08018 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:37.840302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a0945f5a-b518-4b6a-b5f6-0a3e73c08018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:37.866260 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63239ec-1d3e-4cf2-b4fe-7c298ab4f821 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.872070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.873119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.873409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.873680 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] PATH_INFO: `/v3/users/85bdda041ef243f8b77204be2924482f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.874909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=85bdda041ef243f8b77204be2924482f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.876881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.900764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:37.917792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0945f5a-b518-4b6a-b5f6-0a3e73c08018 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:37.925171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:37.926279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:37.926814 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:37.927126 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] PATH_INFO: `/v3/users/0bfc852deaaf41a98919cc338be2e1d7` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:37.928667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfc852deaaf41a98919cc338be2e1d7)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:37.940421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:37.940780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85bdda041ef243f8b77204be2924482f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.949575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85bdda041ef243f8b77204be2924482f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.951123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85bdda041ef243f8b77204be2924482f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.970495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85bdda041ef243f8b77204be2924482f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.988283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85bdda041ef243f8b77204be2924482f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:37.997784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1078/2148] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:37 2022] DELETE /identity/v3/users/85bdda041ef243f8b77204be2924482f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:38.000800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:38.000800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:38.003583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfc852deaaf41a98919cc338be2e1d7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.004408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfc852deaaf41a98919cc338be2e1d7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.012329 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfc852deaaf41a98919cc338be2e1d7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.014286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfc852deaaf41a98919cc338be2e1d7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.028264 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-37681c33-7e21-4b67-a0ef-3bedda5ed166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfc852deaaf41a98919cc338be2e1d7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.043035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1071/2149] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:37 2022] DELETE /identity/v3/users/0bfc852deaaf41a98919cc338be2e1d7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:38.045932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd651f-5662-44f4-9dc1-2606c32477cb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:38.045932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd651f-5662-44f4-9dc1-2606c32477cb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:38.045932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd651f-5662-44f4-9dc1-2606c32477cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:38.063695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-aecd651f-5662-44f4-9dc1-2606c32477cb None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:38.065115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-aecd651f-5662-44f4-9dc1-2606c32477cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:38.114311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572774fd-9984-41c1-814c-2b06556bdbaf admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:38.145511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1072/2150] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:38.155260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd651f-5662-44f4-9dc1-2606c32477cb None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:38.158158 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-aecd651f-5662-44f4-9dc1-2606c32477cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:38.171544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:38.195793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112c515-43aa-4dae-815e-ef2270a0d2b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097b12fbf6494336b11ece5c498a4103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:38.196505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112c515-43aa-4dae-815e-ef2270a0d2b2 service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:38.196969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112c515-43aa-4dae-815e-ef2270a0d2b2 service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:38.197265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112c515-43aa-4dae-815e-ef2270a0d2b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:38.200134 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0112c515-43aa-4dae-815e-ef2270a0d2b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:38.275422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd651f-5662-44f4-9dc1-2606c32477cb None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:38.285259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:38.286501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5aa07e-55d6-4eaf-9f9d-d086e712a2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:38.287187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5aa07e-55d6-4eaf-9f9d-d086e712a2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:38.287470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5aa07e-55d6-4eaf-9f9d-d086e712a2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:38.287780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5aa07e-55d6-4eaf-9f9d-d086e712a2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:38.299609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112c515-43aa-4dae-815e-ef2270a0d2b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:38.301217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112c515-43aa-4dae-815e-ef2270a0d2b2 service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:38.309047 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5aa07e-55d6-4eaf-9f9d-d086e712a2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:38.310540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5aa07e-55d6-4eaf-9f9d-d086e712a2e5 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:38.322517 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1079/2151] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:38.323218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1073/2152] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:38.498790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112c515-43aa-4dae-815e-ef2270a0d2b2 service glance] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:38.500550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0112c515-43aa-4dae-815e-ef2270a0d2b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:38.524673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112c515-43aa-4dae-815e-ef2270a0d2b2 service glance] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:38.540989 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:38.541962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:38.542303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:38.543024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] PATH_INFO: `/v3/projects/8c5997e318844d67beb7afef7ea86e9e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:38.544415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c5997e318844d67beb7afef7ea86e9e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:38.546887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:38.622952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c5997e318844d67beb7afef7ea86e9e', 'request_id': 'req-319cf58c-31b0-400b-bce2-057ffc4bbcc5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.631062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c5997e318844d67beb7afef7ea86e9e', 'request_id': 'req-319cf58c-31b0-400b-bce2-057ffc4bbcc5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.641328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c5997e318844d67beb7afef7ea86e9e', 'request_id': 'req-319cf58c-31b0-400b-bce2-057ffc4bbcc5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.655157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] The token cache is being invalidate because project 8c5997e318844d67beb7afef7ea86e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:38.655598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.656336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.657962 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1080/2153] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:38 2022] DELETE /identity/v3/projects/8c5997e318844d67beb7afef7ea86e9e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:38.662470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5aa07e-55d6-4eaf-9f9d-d086e712a2e5 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:38.674865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5aa07e-55d6-4eaf-9f9d-d086e712a2e5 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:38.716797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:38.718585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:38.762963 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:38.763952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:38.764321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:38.764652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] PATH_INFO: `/v3/users/834df9c33a814dc69af5f9e7eaee930e` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:38.766030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=834df9c33a814dc69af5f9e7eaee930e)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:38.768478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:38.804723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cf58c-31b0-400b-bce2-057ffc4bbcc5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:38.810886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:38.812045 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:38.812419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:38.812808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] PATH_INFO: `/v3/projects/a2335bd0982042e4a907a2fdcc60f869` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:38.814325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2335bd0982042e4a907a2fdcc60f869)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:38.816663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:38.823405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834df9c33a814dc69af5f9e7eaee930e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.824817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834df9c33a814dc69af5f9e7eaee930e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.834373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834df9c33a814dc69af5f9e7eaee930e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.835921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834df9c33a814dc69af5f9e7eaee930e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.851214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834df9c33a814dc69af5f9e7eaee930e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.867959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2335bd0982042e4a907a2fdcc60f869', 'request_id': 'req-e9842be5-b5e1-4864-a95d-b853b2b385c1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.871031 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1074/2154] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:38 2022] DELETE /identity/v3/users/834df9c33a814dc69af5f9e7eaee930e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:38.885095 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2335bd0982042e4a907a2fdcc60f869', 'request_id': 'req-e9842be5-b5e1-4864-a95d-b853b2b385c1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.896178 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2335bd0982042e4a907a2fdcc60f869', 'request_id': 'req-e9842be5-b5e1-4864-a95d-b853b2b385c1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.907782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] The token cache is being invalidate because project a2335bd0982042e4a907a2fdcc60f869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:38.907973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.908885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:38.911119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1081/2155] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:38 2022] DELETE /identity/v3/projects/a2335bd0982042e4a907a2fdcc60f869 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:38.915336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:38.917019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:38.978601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:39.020577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48589ce-b06b-4e56-8768-3471f764e960 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:39.025577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:39.026464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:39.026875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:39.027131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] PATH_INFO: `/v3/users/a42268e606ea44c7b3219da841f517a0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:39.028777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a42268e606ea44c7b3219da841f517a0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:39.030847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:39.084628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a42268e606ea44c7b3219da841f517a0'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.085353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a42268e606ea44c7b3219da841f517a0'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.092411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a42268e606ea44c7b3219da841f517a0'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.093066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a42268e606ea44c7b3219da841f517a0'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.105198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a42268e606ea44c7b3219da841f517a0'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.119350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1075/2156] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:38 2022] DELETE /identity/v3/users/a42268e606ea44c7b3219da841f517a0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:39.207631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:39.209336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:39.216553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:39.301077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9842be5-b5e1-4864-a95d-b853b2b385c1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:39.366583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:39.367733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:39.368150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:39.368608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] PATH_INFO: `/v3/projects/419613ac24fb4a8bb0b190c5c101f532` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:39.370168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=419613ac24fb4a8bb0b190c5c101f532)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:39.372204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:39.423970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:39.428336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:39.441254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '419613ac24fb4a8bb0b190c5c101f532', 'request_id': 'req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.448885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '419613ac24fb4a8bb0b190c5c101f532', 'request_id': 'req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.458860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c06eb-5fa2-4685-bbc2-cb7a75cda900 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:39.462756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '419613ac24fb4a8bb0b190c5c101f532', 'request_id': 'req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.480599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] The token cache is being invalidate because project 419613ac24fb4a8bb0b190c5c101f532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:39.480965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.482202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.488808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1082/2157] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:39 2022] DELETE /identity/v3/projects/419613ac24fb4a8bb0b190c5c101f532 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:39.590330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:39.593100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:39.594027 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:39.594307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:39.594642 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] PATH_INFO: `/v3/projects/68b21a03c91b41ba8295de67c7606957` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:39.596545 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:39.597161 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68b21a03c91b41ba8295de67c7606957)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:39.599817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:39.616974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f286da5-e5b3-4119-a9f1-509a1e9c9b6f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:39.624645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:39.626592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:39.627245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:39.627840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] PATH_INFO: `/v3/users/c56e3bcc509f4d4abc860b954be44380` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:39.630026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c56e3bcc509f4d4abc860b954be44380)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:39.632485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:39.696870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:39.711559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68b21a03c91b41ba8295de67c7606957', 'request_id': 'req-734eebbf-bbf0-418a-aa4a-d52f39376076'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.725115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c56e3bcc509f4d4abc860b954be44380'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.727137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:39.733442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68b21a03c91b41ba8295de67c7606957', 'request_id': 'req-734eebbf-bbf0-418a-aa4a-d52f39376076'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.736942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c56e3bcc509f4d4abc860b954be44380'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.738025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c56e3bcc509f4d4abc860b954be44380'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.743908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68b21a03c91b41ba8295de67c7606957', 'request_id': 'req-734eebbf-bbf0-418a-aa4a-d52f39376076'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.746881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c56e3bcc509f4d4abc860b954be44380'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.756021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] The token cache is being invalidate because project 68b21a03c91b41ba8295de67c7606957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:39.756330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.756995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.758664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1076/2158] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:39 2022] DELETE /identity/v3/projects/68b21a03c91b41ba8295de67c7606957 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:39.760709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6d975936-37d9-4352-9534-c4b828a59424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c56e3bcc509f4d4abc860b954be44380'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.762625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:39.764555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:39.770291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1083/2159] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:39 2022] DELETE /identity/v3/users/c56e3bcc509f4d4abc860b954be44380 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:39.775185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344d7fb-02d0-4215-b8c3-ac2c821333de None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:39.775556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344d7fb-02d0-4215-b8c3-ac2c821333de None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:39.775907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344d7fb-02d0-4215-b8c3-ac2c821333de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:39.796796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4344d7fb-02d0-4215-b8c3-ac2c821333de None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:39.798247 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4344d7fb-02d0-4215-b8c3-ac2c821333de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:39.846503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734eebbf-bbf0-418a-aa4a-d52f39376076 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:39.870137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1084/2160] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:39.880497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344d7fb-02d0-4215-b8c3-ac2c821333de None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:39.882342 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4344d7fb-02d0-4215-b8c3-ac2c821333de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:39.912768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:39.913665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:39.913943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:39.914212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] PATH_INFO: `/v3/users/598a4e01921b40548fff2ded1148fbef` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:39.915431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=598a4e01921b40548fff2ded1148fbef)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:39.917621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:39.958469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344d7fb-02d0-4215-b8c3-ac2c821333de None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:39.962833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a2d50-30f2-4656-a727-e1ae661c079b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:39.963493 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a2d50-30f2-4656-a727-e1ae661c079b service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:39.963777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a2d50-30f2-4656-a727-e1ae661c079b service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:39.964047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a2d50-30f2-4656-a727-e1ae661c079b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:39.975484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '598a4e01921b40548fff2ded1148fbef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.976401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '598a4e01921b40548fff2ded1148fbef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.979056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a2d50-30f2-4656-a727-e1ae661c079b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:39.980597 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a2d50-30f2-4656-a727-e1ae661c079b service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:39.989816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '598a4e01921b40548fff2ded1148fbef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.989816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '598a4e01921b40548fff2ded1148fbef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:39.993033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1085/2161] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:40.019085 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '598a4e01921b40548fff2ded1148fbef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.035844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1077/2162] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:39 2022] DELETE /identity/v3/users/598a4e01921b40548fff2ded1148fbef => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:40.459879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a2d50-30f2-4656-a727-e1ae661c079b service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:40.461388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-246a2d50-30f2-4656-a727-e1ae661c079b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:40.483940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a2d50-30f2-4656-a727-e1ae661c079b service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:40.527194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:40.530202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:40.542467 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:40.563774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:40.570805 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:40.571804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:40.572417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:40.572771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] PATH_INFO: `/v3/projects/0c934c24022c47ae9c73389fffe022fa` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:40.575423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c934c24022c47ae9c73389fffe022fa)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:40.576738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:40.606952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d696f-56e9-4c10-b5d1-6ee59923f9cc admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:40.612361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:40.613273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:40.613572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:40.613826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] PATH_INFO: `/v3/projects/fa3f715ae9d0465ba3b90a8d0cd13bf2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:40.615131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa3f715ae9d0465ba3b90a8d0cd13bf2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:40.617151 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:40.628738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c934c24022c47ae9c73389fffe022fa', 'request_id': 'req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.636393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c934c24022c47ae9c73389fffe022fa', 'request_id': 'req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.646525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c934c24022c47ae9c73389fffe022fa', 'request_id': 'req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.659070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] The token cache is being invalidate because project 0c934c24022c47ae9c73389fffe022fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:40.659430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.660483 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.662952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1086/2163] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:40 2022] DELETE /identity/v3/projects/0c934c24022c47ae9c73389fffe022fa => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:40.666978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:40.668563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:40.673712 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa3f715ae9d0465ba3b90a8d0cd13bf2', 'request_id': 'req-42beeec1-4e7a-4a23-a0b0-094b28f14945'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.680459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa3f715ae9d0465ba3b90a8d0cd13bf2', 'request_id': 'req-42beeec1-4e7a-4a23-a0b0-094b28f14945'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.689388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa3f715ae9d0465ba3b90a8d0cd13bf2', 'request_id': 'req-42beeec1-4e7a-4a23-a0b0-094b28f14945'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.701024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] The token cache is being invalidate because project fa3f715ae9d0465ba3b90a8d0cd13bf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:40.701421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.702137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42beeec1-4e7a-4a23-a0b0-094b28f14945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.703800 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1078/2164] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:40 2022] DELETE /identity/v3/projects/fa3f715ae9d0465ba3b90a8d0cd13bf2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:40.709119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba389a-35e4-475d-88e7-2f96004145a0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:40.709401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba389a-35e4-475d-88e7-2f96004145a0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:40.709700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba389a-35e4-475d-88e7-2f96004145a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:40.727454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-21ba389a-35e4-475d-88e7-2f96004145a0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:40.729635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-21ba389a-35e4-475d-88e7-2f96004145a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:40.735295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cffb7-03e4-4a1d-8f9b-7a58481111d5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:40.802408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:40.803275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:40.803563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:40.803835 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] PATH_INFO: `/v3/users/9f95b306df78477d9521d5304195ce69` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:40.805135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f95b306df78477d9521d5304195ce69)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:40.807537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:40.808728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1079/2165] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:40.818749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2464270-2aa9-4231-a55a-af83e4a04131 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:40.818749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2464270-2aa9-4231-a55a-af83e4a04131 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:40.818749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2464270-2aa9-4231-a55a-af83e4a04131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:40.839244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c2464270-2aa9-4231-a55a-af83e4a04131 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:40.840529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c2464270-2aa9-4231-a55a-af83e4a04131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:40.852149 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1080/2166] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:40.855064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2464270-2aa9-4231-a55a-af83e4a04131 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:40.863854 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:40.864945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2464270-2aa9-4231-a55a-af83e4a04131 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:40.869810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d2a11-1d36-4bc5-ae2d-5812eb1d99fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:40.870492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d2a11-1d36-4bc5-ae2d-5812eb1d99fd admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:40.870799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d2a11-1d36-4bc5-ae2d-5812eb1d99fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:40.871038 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d2a11-1d36-4bc5-ae2d-5812eb1d99fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:40.871808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d2a11-1d36-4bc5-ae2d-5812eb1d99fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:40.872149 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d2a11-1d36-4bc5-ae2d-5812eb1d99fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:40.873837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d2a11-1d36-4bc5-ae2d-5812eb1d99fd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:40.876779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f95b306df78477d9521d5304195ce69'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.892175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1081/2167] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:58:40.892175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d2a11-1d36-4bc5-ae2d-5812eb1d99fd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:40.897959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f95b306df78477d9521d5304195ce69'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.898644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f95b306df78477d9521d5304195ce69'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.905308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d2a11-1d36-4bc5-ae2d-5812eb1d99fd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:40.908468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f95b306df78477d9521d5304195ce69'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.911411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b476e6-2024-4c71-8f2f-9b513d4e3116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:40.912052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b476e6-2024-4c71-8f2f-9b513d4e3116 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:40.912418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b476e6-2024-4c71-8f2f-9b513d4e3116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:40.912751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b476e6-2024-4c71-8f2f-9b513d4e3116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:40.913514 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b476e6-2024-4c71-8f2f-9b513d4e3116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:40.923292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b476e6-2024-4c71-8f2f-9b513d4e3116 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:40.923419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f95b306df78477d9521d5304195ce69'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:40.929546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1087/2168] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:40 2022] DELETE /identity/v3/users/9f95b306df78477d9521d5304195ce69 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:40.933112 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:40.946671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1082/2169] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:40 2022] POST /identity/v3/projects => generated 389 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:40.950959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:40.952457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:41.032996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c76000-f262-4e04-8ecd-b623e6290246 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:41.105647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356560ee-5ea0-4dcb-b087-ff1b79471b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:41.106397 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356560ee-5ea0-4dcb-b087-ff1b79471b69 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:41.106682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356560ee-5ea0-4dcb-b087-ff1b79471b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:41.106955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356560ee-5ea0-4dcb-b087-ff1b79471b69 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:41.107668 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356560ee-5ea0-4dcb-b087-ff1b79471b69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:41.109620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356560ee-5ea0-4dcb-b087-ff1b79471b69 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:41.135567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1088/2170] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:40 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:41.136888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b476e6-2024-4c71-8f2f-9b513d4e3116 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:41.150530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b476e6-2024-4c71-8f2f-9b513d4e3116 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:41.160198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968b0f4-e488-4c1d-b892-41a7eafb1b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:41.160198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968b0f4-e488-4c1d-b892-41a7eafb1b4f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:41.160198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968b0f4-e488-4c1d-b892-41a7eafb1b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:41.160198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968b0f4-e488-4c1d-b892-41a7eafb1b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:41.160533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968b0f4-e488-4c1d-b892-41a7eafb1b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:41.161176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968b0f4-e488-4c1d-b892-41a7eafb1b4f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:41.172157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1083/2171] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:41.176784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356560ee-5ea0-4dcb-b087-ff1b79471b69 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:41.192405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356560ee-5ea0-4dcb-b087-ff1b79471b69 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:41.197559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dc05c-c7ae-462d-a43f-59e9254ade1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:41.198241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dc05c-c7ae-462d-a43f-59e9254ade1a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:41.198547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dc05c-c7ae-462d-a43f-59e9254ade1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:41.198846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dc05c-c7ae-462d-a43f-59e9254ade1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:41.199599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dc05c-c7ae-462d-a43f-59e9254ade1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:41.201341 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dc05c-c7ae-462d-a43f-59e9254ade1a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:41.213525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1089/2172] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:41.217967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968b0f4-e488-4c1d-b892-41a7eafb1b4f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:41.227943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dc05c-c7ae-462d-a43f-59e9254ade1a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:41.229251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968b0f4-e488-4c1d-b892-41a7eafb1b4f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:41.229715 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-314dc05c-c7ae-462d-a43f-59e9254ade1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:41.234582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438682e-2aba-4861-a9f4-d8d595854f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:41.235616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438682e-2aba-4861-a9f4-d8d595854f7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:41.235893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438682e-2aba-4861-a9f4-d8d595854f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:41.236204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438682e-2aba-4861-a9f4-d8d595854f7c admin admin] PATH_INFO: `/v3/projects/a3498bff28ef4749b0b1444c29b58717/users/82a2e011675540258677296d77d12bd9/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:41.247292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dc05c-c7ae-462d-a43f-59e9254ade1a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:41.253927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:41.255133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:41.255416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:41.255726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] PATH_INFO: `/v3/projects/b9ebb7f483a84ea5beab691ce0144060` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:41.257015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9ebb7f483a84ea5beab691ce0144060)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:41.259131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:41.262293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438682e-2aba-4861-a9f4-d8d595854f7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3498bff28ef4749b0b1444c29b58717, user_id=82a2e011675540258677296d77d12bd9, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:41.265088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438682e-2aba-4861-a9f4-d8d595854f7c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:41.279263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1084/2173] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:41 2022] PUT /identity/v3/projects/a3498bff28ef4749b0b1444c29b58717/users/82a2e011675540258677296d77d12bd9/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:41.290613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ec9f7-e5f7-4619-a757-4ef2a619607a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:41.290897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ec9f7-e5f7-4619-a757-4ef2a619607a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:41.291178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ec9f7-e5f7-4619-a757-4ef2a619607a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:41.307909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:41.321877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9ebb7f483a84ea5beab691ce0144060', 'request_id': 'req-cc44b732-1588-4ea8-9146-201693409df4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.323018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5f2ec9f7-e5f7-4619-a757-4ef2a619607a None None] MFA Rules not processed for user `82a2e011675540258677296d77d12bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:41.324188 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5f2ec9f7-e5f7-4619-a757-4ef2a619607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:41.329056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9ebb7f483a84ea5beab691ce0144060', 'request_id': 'req-cc44b732-1588-4ea8-9146-201693409df4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.338951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9ebb7f483a84ea5beab691ce0144060', 'request_id': 'req-cc44b732-1588-4ea8-9146-201693409df4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.350299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] The token cache is being invalidate because project b9ebb7f483a84ea5beab691ce0144060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:41.350584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.351239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.353283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1090/2174] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:41 2022] DELETE /identity/v3/projects/b9ebb7f483a84ea5beab691ce0144060 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:41.357271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:41.358235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:41.391625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1085/2175] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:58:41.402600 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ec9f7-e5f7-4619-a757-4ef2a619607a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:41.403991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5f2ec9f7-e5f7-4619-a757-4ef2a619607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:41.422318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44b732-1588-4ea8-9146-201693409df4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:41.427352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:41.428322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:41.428607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:41.428882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] PATH_INFO: `/v3/users/3bda5a3396e54d07beae222af0183c9b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:41.430225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bda5a3396e54d07beae222af0183c9b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:41.432590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:41.465677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ec9f7-e5f7-4619-a757-4ef2a619607a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:41.469313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54ff29-8a85-4265-86c4-ee5ec7c0e308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:41.469966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54ff29-8a85-4265-86c4-ee5ec7c0e308 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:41.470243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54ff29-8a85-4265-86c4-ee5ec7c0e308 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:41.470543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54ff29-8a85-4265-86c4-ee5ec7c0e308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:41.479560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54ff29-8a85-4265-86c4-ee5ec7c0e308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:41.481042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54ff29-8a85-4265-86c4-ee5ec7c0e308 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:41.482044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bda5a3396e54d07beae222af0183c9b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.489044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bda5a3396e54d07beae222af0183c9b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.489746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bda5a3396e54d07beae222af0183c9b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.490768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1086/2176] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:58:41.498166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bda5a3396e54d07beae222af0183c9b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.511108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c7f06d80-33e0-4b13-9e07-8abed9e5fe59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bda5a3396e54d07beae222af0183c9b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.517529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1091/2177] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:58:41 2022] DELETE /identity/v3/users/3bda5a3396e54d07beae222af0183c9b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:58:41.832941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54ff29-8a85-4265-86c4-ee5ec7c0e308 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:41.853463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54ff29-8a85-4265-86c4-ee5ec7c0e308 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:41.915061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:41.926968 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:41.927949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:41.928326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:41.928604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] PATH_INFO: `/v3/projects/68ad6fc409374a77a5ae0602f2213168` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:41.929933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68ad6fc409374a77a5ae0602f2213168)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:41.932312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:41.983958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ad6fc409374a77a5ae0602f2213168', 'request_id': 'req-3814aadf-9082-4b76-8c1e-17514a878a59'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.990456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ad6fc409374a77a5ae0602f2213168', 'request_id': 'req-3814aadf-9082-4b76-8c1e-17514a878a59'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:41.998732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ad6fc409374a77a5ae0602f2213168', 'request_id': 'req-3814aadf-9082-4b76-8c1e-17514a878a59'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:42.009727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] The token cache is being invalidate because project 68ad6fc409374a77a5ae0602f2213168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:58:42.010033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:42.010660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3814aadf-9082-4b76-8c1e-17514a878a59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:58:42.012267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1087/2178] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:58:41 2022] DELETE /identity/v3/projects/68ad6fc409374a77a5ae0602f2213168 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:42.017104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13046f-61aa-4531-80ce-6ef517ce46b4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:42.017630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13046f-61aa-4531-80ce-6ef517ce46b4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:42.017792 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13046f-61aa-4531-80ce-6ef517ce46b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:42.035509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3f13046f-61aa-4531-80ce-6ef517ce46b4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:42.036771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3f13046f-61aa-4531-80ce-6ef517ce46b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:42.078822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:42.111137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1092/2179] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:42.118947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a2ed-b5e8-461d-9040-bca84e789ba5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:42.119239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a2ed-b5e8-461d-9040-bca84e789ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:42.119515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a2ed-b5e8-461d-9040-bca84e789ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:42.140098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6992a2ed-b5e8-461d-9040-bca84e789ba5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:42.141341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6992a2ed-b5e8-461d-9040-bca84e789ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:42.149986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1088/2180] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:42.154837 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13046f-61aa-4531-80ce-6ef517ce46b4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:42.167421 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13046f-61aa-4531-80ce-6ef517ce46b4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:42.172671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e35838-6a04-433d-904e-678b6d3d458b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:42.173365 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e35838-6a04-433d-904e-678b6d3d458b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:42.173644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e35838-6a04-433d-904e-678b6d3d458b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:42.174041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e35838-6a04-433d-904e-678b6d3d458b admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:42.174971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e35838-6a04-433d-904e-678b6d3d458b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:58:42.175282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e35838-6a04-433d-904e-678b6d3d458b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:42.176924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e35838-6a04-433d-904e-678b6d3d458b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:42.190318 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1093/2181] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:58:42.195052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992a2ed-b5e8-461d-9040-bca84e789ba5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:42.204177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992a2ed-b5e8-461d-9040-bca84e789ba5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:42.209200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6b24e-9e3e-4524-b117-fca0a9ba65d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:42.209860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6b24e-9e3e-4524-b117-fca0a9ba65d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:42.210141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6b24e-9e3e-4524-b117-fca0a9ba65d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:42.210415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6b24e-9e3e-4524-b117-fca0a9ba65d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:42.211123 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6b24e-9e3e-4524-b117-fca0a9ba65d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:42.213077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6b24e-9e3e-4524-b117-fca0a9ba65d6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:42.239904 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1089/2182] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:58:42 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:42.244248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e35838-6a04-433d-904e-678b6d3d458b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:42.255270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e35838-6a04-433d-904e-678b6d3d458b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:42.316695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef2866-27ba-4ba1-8694-60a30a40497d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:42.317379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef2866-27ba-4ba1-8694-60a30a40497d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:42.317657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef2866-27ba-4ba1-8694-60a30a40497d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:42.317928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef2866-27ba-4ba1-8694-60a30a40497d admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:42.318620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef2866-27ba-4ba1-8694-60a30a40497d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:42.320637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef2866-27ba-4ba1-8694-60a30a40497d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:42.341427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1094/2183] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:58:42 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:58:42.346038 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6b24e-9e3e-4524-b117-fca0a9ba65d6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:42.358082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6b24e-9e3e-4524-b117-fca0a9ba65d6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:42.363872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c28b4-8d69-4546-82aa-3ae0e9925f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:42.364609 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c28b4-8d69-4546-82aa-3ae0e9925f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:42.364889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c28b4-8d69-4546-82aa-3ae0e9925f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:42.365163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c28b4-8d69-4546-82aa-3ae0e9925f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:42.365919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c28b4-8d69-4546-82aa-3ae0e9925f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:42.367533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c28b4-8d69-4546-82aa-3ae0e9925f8d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:42.379168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1090/2184] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:42.384030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef2866-27ba-4ba1-8694-60a30a40497d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:42.395291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef2866-27ba-4ba1-8694-60a30a40497d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:42.400981 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf8ca6-d884-46bb-85bf-8d41eb64c948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:42.401747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf8ca6-d884-46bb-85bf-8d41eb64c948 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:42.402119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf8ca6-d884-46bb-85bf-8d41eb64c948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:42.402463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf8ca6-d884-46bb-85bf-8d41eb64c948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:42.403257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf8ca6-d884-46bb-85bf-8d41eb64c948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:42.405024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf8ca6-d884-46bb-85bf-8d41eb64c948 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:42.419512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1095/2185] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:58:42.420837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c28b4-8d69-4546-82aa-3ae0e9925f8d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:42.433650 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c28b4-8d69-4546-82aa-3ae0e9925f8d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:42.438346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee56cc-d8f5-4e98-a7a7-39f8ef1ece19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:42.439257 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee56cc-d8f5-4e98-a7a7-39f8ef1ece19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:42.439605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee56cc-d8f5-4e98-a7a7-39f8ef1ece19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:42.440031 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee56cc-d8f5-4e98-a7a7-39f8ef1ece19 admin admin] PATH_INFO: `/v3/projects/804c8a0ad1674b818f8bbc53d661284b/users/934b314c46824d5397a0ae92bb07b725/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:42.455736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee56cc-d8f5-4e98-a7a7-39f8ef1ece19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=804c8a0ad1674b818f8bbc53d661284b, user_id=934b314c46824d5397a0ae92bb07b725, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:42.458736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee56cc-d8f5-4e98-a7a7-39f8ef1ece19 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:42.474960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1091/2186] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:58:42 2022] PUT /identity/v3/projects/804c8a0ad1674b818f8bbc53d661284b/users/934b314c46824d5397a0ae92bb07b725/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:58:42.483182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122923-7cbd-48bf-b5f4-a3bd815ac453 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:42.483491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122923-7cbd-48bf-b5f4-a3bd815ac453 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:42.483780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122923-7cbd-48bf-b5f4-a3bd815ac453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:42.503509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-62122923-7cbd-48bf-b5f4-a3bd815ac453 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:42.504778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-62122923-7cbd-48bf-b5f4-a3bd815ac453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:42.569105 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1096/2187] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:42.578812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee56cc-d8f5-4e98-a7a7-39f8ef1ece19 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:42.580677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b4ee56cc-d8f5-4e98-a7a7-39f8ef1ece19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:42.641634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee56cc-d8f5-4e98-a7a7-39f8ef1ece19 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:42.645655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df079279-8ce5-4ee7-b222-163632b2a233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:42.646306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df079279-8ce5-4ee7-b222-163632b2a233 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:42.646587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df079279-8ce5-4ee7-b222-163632b2a233 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:42.646859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df079279-8ce5-4ee7-b222-163632b2a233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:42.656494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df079279-8ce5-4ee7-b222-163632b2a233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:42.657937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df079279-8ce5-4ee7-b222-163632b2a233 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:42.667935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1092/2188] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:43.605836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce393fdb-af94-49c5-8808-ac5a59f6cab9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:43.606596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce393fdb-af94-49c5-8808-ac5a59f6cab9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:43.606596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce393fdb-af94-49c5-8808-ac5a59f6cab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:43.643989 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ce393fdb-af94-49c5-8808-ac5a59f6cab9 None None] MFA Rules not processed for user `934b314c46824d5397a0ae92bb07b725`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:43.645169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ce393fdb-af94-49c5-8808-ac5a59f6cab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:43.698568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:43.728491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1097/2189] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:43.740289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df079279-8ce5-4ee7-b222-163632b2a233 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:58:43.741802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-df079279-8ce5-4ee7-b222-163632b2a233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:43.768636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:58:43.803130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df079279-8ce5-4ee7-b222-163632b2a233 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:58:43.807620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c87dd-41e7-4756-9cfc-51a96dae330b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:58:43.808317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c87dd-41e7-4756-9cfc-51a96dae330b service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:43.808596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c87dd-41e7-4756-9cfc-51a96dae330b service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:43.808863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c87dd-41e7-4756-9cfc-51a96dae330b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:43.818292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c87dd-41e7-4756-9cfc-51a96dae330b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:58:43.819762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c87dd-41e7-4756-9cfc-51a96dae330b service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:58:43.829100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1093/2190] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:58:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:58:50.449808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cb3b9-ef22-495c-9429-71e6a7b7ff26 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:58:50.450308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cb3b9-ef22-495c-9429-71e6a7b7ff26 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:58:50.450308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cb3b9-ef22-495c-9429-71e6a7b7ff26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:58:50.468861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6e1cb3b9-ef22-495c-9429-71e6a7b7ff26 None None] MFA Rules not processed for user `934b314c46824d5397a0ae92bb07b725`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:58:50.470055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6e1cb3b9-ef22-495c-9429-71e6a7b7ff26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:58:50.478737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1098/2191] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:58:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:58:53.824927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=16 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:00.463319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=13 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:10.695362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249f5a1-6d30-46d2-b894-2d03878e6772 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:10.695936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249f5a1-6d30-46d2-b894-2d03878e6772 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:10.696127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249f5a1-6d30-46d2-b894-2d03878e6772 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:10.697715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1094/2192] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 20:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 20:59:10.705002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456903c2-a4d2-4e9a-8f70-59da48f08a7c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:10.705458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456903c2-a4d2-4e9a-8f70-59da48f08a7c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:10.705582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456903c2-a4d2-4e9a-8f70-59da48f08a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:10.724745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-456903c2-a4d2-4e9a-8f70-59da48f08a7c None None] MFA Rules not processed for user `946d11b552754364b55eb495f68ab852`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:10.725956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-456903c2-a4d2-4e9a-8f70-59da48f08a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:10.767864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249f5a1-6d30-46d2-b894-2d03878e6772 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:10.769845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3249f5a1-6d30-46d2-b894-2d03878e6772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:10.786574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249f5a1-6d30-46d2-b894-2d03878e6772 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:10.791130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:10.792045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:10.792405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:10.792685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] PATH_INFO: `/v3/users/b5871ced3beb4decacc3a20afe523349` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:10.793948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5871ced3beb4decacc3a20afe523349)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:10.795939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:10.800751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1099/2193] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 20:59:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:59:10.812467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456903c2-a4d2-4e9a-8f70-59da48f08a7c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:10.813966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-456903c2-a4d2-4e9a-8f70-59da48f08a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:10.832695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456903c2-a4d2-4e9a-8f70-59da48f08a7c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:10.832695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facde227-e86e-4b66-8714-684ec7455649 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:10.833093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facde227-e86e-4b66-8714-684ec7455649 service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:10.833466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facde227-e86e-4b66-8714-684ec7455649 service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:10.833638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facde227-e86e-4b66-8714-684ec7455649 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:10.842945 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facde227-e86e-4b66-8714-684ec7455649 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:10.844588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facde227-e86e-4b66-8714-684ec7455649 service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:10.855079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1100/2194] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:59:10.855926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5871ced3beb4decacc3a20afe523349'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:10.857003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5871ced3beb4decacc3a20afe523349'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:10.863933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5871ced3beb4decacc3a20afe523349'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:10.864644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5871ced3beb4decacc3a20afe523349'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:10.877039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d74f4995-c607-4ef9-91b8-3604a3c38c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5871ced3beb4decacc3a20afe523349'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:10.891377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1095/2195] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:59:10 2022] DELETE /identity/v3/users/b5871ced3beb4decacc3a20afe523349 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:11.134835 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facde227-e86e-4b66-8714-684ec7455649 service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:11.146780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facde227-e86e-4b66-8714-684ec7455649 service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:11.224019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:11.224930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.225209 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.225481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] PATH_INFO: `/v3/projects/dab2ba718dad487fa0f174b8b0964286` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.226687 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=dab2ba718dad487fa0f174b8b0964286)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:11.228642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:11.274976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dab2ba718dad487fa0f174b8b0964286', 'request_id': 'req-da05cc4b-72de-4c4f-90ae-0f2896e319ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:11.281914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dab2ba718dad487fa0f174b8b0964286', 'request_id': 'req-da05cc4b-72de-4c4f-90ae-0f2896e319ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:11.290009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dab2ba718dad487fa0f174b8b0964286', 'request_id': 'req-da05cc4b-72de-4c4f-90ae-0f2896e319ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:11.301338 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] The token cache is being invalidate because project dab2ba718dad487fa0f174b8b0964286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:59:11.301646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:11.302299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:11.303902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1101/2196] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:59:11 2022] DELETE /identity/v3/projects/dab2ba718dad487fa0f174b8b0964286 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:59:11.309231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f35650-7dbd-46fb-998a-5da41e1fab5e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.309704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f35650-7dbd-46fb-998a-5da41e1fab5e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.310612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f35650-7dbd-46fb-998a-5da41e1fab5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.329797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d2f35650-7dbd-46fb-998a-5da41e1fab5e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:11.330975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d2f35650-7dbd-46fb-998a-5da41e1fab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:11.346944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:11.348387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:11.362554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:11.394549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1096/2197] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:59:11.403250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf521a3-8e78-44e2-9a0f-8cd17b56e4f8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.403614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf521a3-8e78-44e2-9a0f-8cd17b56e4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.404439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf521a3-8e78-44e2-9a0f-8cd17b56e4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.418092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05cc4b-72de-4c4f-90ae-0f2896e319ef admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:11.422486 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9bf521a3-8e78-44e2-9a0f-8cd17b56e4f8 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:11.422983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:11.423696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9bf521a3-8e78-44e2-9a0f-8cd17b56e4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:11.423869 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.424179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.424454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] PATH_INFO: `/v3/users/82a2e011675540258677296d77d12bd9` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.425659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82a2e011675540258677296d77d12bd9)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:11.427625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:11.432002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1097/2198] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:59:11.436803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf521a3-8e78-44e2-9a0f-8cd17b56e4f8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:11.446032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf521a3-8e78-44e2-9a0f-8cd17b56e4f8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:11.450699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0133e-0d78-4991-831c-4dca1c74e50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:11.451346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0133e-0d78-4991-831c-4dca1c74e50b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.451625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0133e-0d78-4991-831c-4dca1c74e50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.451935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0133e-0d78-4991-831c-4dca1c74e50b admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.452719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0133e-0d78-4991-831c-4dca1c74e50b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:59:11.453029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0133e-0d78-4991-831c-4dca1c74e50b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:11.454651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0133e-0d78-4991-831c-4dca1c74e50b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:11.468147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1098/2199] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 20:59:11.471171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0133e-0d78-4991-831c-4dca1c74e50b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:11.480830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82a2e011675540258677296d77d12bd9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:11.481200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0133e-0d78-4991-831c-4dca1c74e50b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:11.485927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef6507-bfe7-4f06-8831-6950f4bb2836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:11.486565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef6507-bfe7-4f06-8831-6950f4bb2836 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.486873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef6507-bfe7-4f06-8831-6950f4bb2836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.487142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef6507-bfe7-4f06-8831-6950f4bb2836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.487874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef6507-bfe7-4f06-8831-6950f4bb2836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:11.488335 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82a2e011675540258677296d77d12bd9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:11.489026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82a2e011675540258677296d77d12bd9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:11.489733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef6507-bfe7-4f06-8831-6950f4bb2836 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:11.497569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82a2e011675540258677296d77d12bd9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:11.510718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d913710e-abb2-4c63-9cb9-a9be76b63b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82a2e011675540258677296d77d12bd9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:11.516063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1099/2200] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:59:11 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:59:11.516999 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1102/2201] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:59:11 2022] DELETE /identity/v3/users/82a2e011675540258677296d77d12bd9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:59:11.520345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef6507-bfe7-4f06-8831-6950f4bb2836 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:11.521129 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55366a-f416-45cf-b5b2-7adeebd3a877 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.521415 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55366a-f416-45cf-b5b2-7adeebd3a877 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.521716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55366a-f416-45cf-b5b2-7adeebd3a877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.521979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-65ef6507-bfe7-4f06-8831-6950f4bb2836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:11.539216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2b55366a-f416-45cf-b5b2-7adeebd3a877 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:11.540430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2b55366a-f416-45cf-b5b2-7adeebd3a877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:11.554190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:11.587704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef6507-bfe7-4f06-8831-6950f4bb2836 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:11.603596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1103/2202] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:59:11.614254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55366a-f416-45cf-b5b2-7adeebd3a877 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:11.616503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2b55366a-f416-45cf-b5b2-7adeebd3a877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:11.666277 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:11.669678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6405d5-915d-4c97-8468-7db8debb8489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:11.670139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6405d5-915d-4c97-8468-7db8debb8489 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.670422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6405d5-915d-4c97-8468-7db8debb8489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.670701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6405d5-915d-4c97-8468-7db8debb8489 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.671644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6405d5-915d-4c97-8468-7db8debb8489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:11.673389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6405d5-915d-4c97-8468-7db8debb8489 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:11.693436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55366a-f416-45cf-b5b2-7adeebd3a877 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:11.696536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1100/2203] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:59:11 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:11.700227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915797b7-a41e-4d69-a84c-9b0f1abc038f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:11.700418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6405d5-915d-4c97-8468-7db8debb8489 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:11.700895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915797b7-a41e-4d69-a84c-9b0f1abc038f service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.701169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915797b7-a41e-4d69-a84c-9b0f1abc038f service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.701441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915797b7-a41e-4d69-a84c-9b0f1abc038f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.710616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6405d5-915d-4c97-8468-7db8debb8489 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:11.711359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915797b7-a41e-4d69-a84c-9b0f1abc038f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:11.713802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915797b7-a41e-4d69-a84c-9b0f1abc038f service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:11.716665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328daecd-001c-4016-8490-a3dd1fb0048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:11.717496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328daecd-001c-4016-8490-a3dd1fb0048b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.717883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328daecd-001c-4016-8490-a3dd1fb0048b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.718253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328daecd-001c-4016-8490-a3dd1fb0048b admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.719103 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328daecd-001c-4016-8490-a3dd1fb0048b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:11.720966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328daecd-001c-4016-8490-a3dd1fb0048b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:11.724274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1104/2204] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:59:11.733189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1101/2205] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:59:11.738047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915797b7-a41e-4d69-a84c-9b0f1abc038f service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:11.747349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915797b7-a41e-4d69-a84c-9b0f1abc038f service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:11.752139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeec1c6-ec73-40be-ac77-e48f4b36a434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:11.752799 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeec1c6-ec73-40be-ac77-e48f4b36a434 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.753083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeec1c6-ec73-40be-ac77-e48f4b36a434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.753356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeec1c6-ec73-40be-ac77-e48f4b36a434 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.754087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeec1c6-ec73-40be-ac77-e48f4b36a434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:11.755693 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeec1c6-ec73-40be-ac77-e48f4b36a434 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:11.769553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1105/2206] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:59:11.770841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328daecd-001c-4016-8490-a3dd1fb0048b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:11.781542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328daecd-001c-4016-8490-a3dd1fb0048b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:11.787048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62063852-7cbb-40d8-8496-70398086a42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:11.787951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62063852-7cbb-40d8-8496-70398086a42c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.788266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62063852-7cbb-40d8-8496-70398086a42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.788560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62063852-7cbb-40d8-8496-70398086a42c admin admin] PATH_INFO: `/v3/projects/cabe41bcf26c41eb8b32910445415ea0/users/e32254f1275a4877af748c721ead0636/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.805979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62063852-7cbb-40d8-8496-70398086a42c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cabe41bcf26c41eb8b32910445415ea0, user_id=e32254f1275a4877af748c721ead0636, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:11.808717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62063852-7cbb-40d8-8496-70398086a42c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:11.823295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1102/2207] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:59:11 2022] PUT /identity/v3/projects/cabe41bcf26c41eb8b32910445415ea0/users/e32254f1275a4877af748c721ead0636/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:11.835001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302a3d0-4e94-4caf-adf4-e6b88e3c8684 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:11.835293 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302a3d0-4e94-4caf-adf4-e6b88e3c8684 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:11.835692 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302a3d0-4e94-4caf-adf4-e6b88e3c8684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:11.862943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d302a3d0-4e94-4caf-adf4-e6b88e3c8684 None None] MFA Rules not processed for user `e32254f1275a4877af748c721ead0636`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:11.864183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d302a3d0-4e94-4caf-adf4-e6b88e3c8684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:11.937209 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1106/2208] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:11.942005 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62063852-7cbb-40d8-8496-70398086a42c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:12.010307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302a3d0-4e94-4caf-adf4-e6b88e3c8684 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:12.011458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62063852-7cbb-40d8-8496-70398086a42c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:12.011748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d302a3d0-4e94-4caf-adf4-e6b88e3c8684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:12.017614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2853a15-a5b1-4614-9ef4-3ce3b6be2655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:12.018348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2853a15-a5b1-4614-9ef4-3ce3b6be2655 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:12.018630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2853a15-a5b1-4614-9ef4-3ce3b6be2655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:12.018903 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2853a15-a5b1-4614-9ef4-3ce3b6be2655 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:12.019714 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2853a15-a5b1-4614-9ef4-3ce3b6be2655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:12.021791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2853a15-a5b1-4614-9ef4-3ce3b6be2655 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:12.030935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302a3d0-4e94-4caf-adf4-e6b88e3c8684 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:12.037232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:12.038420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:12.038967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:12.038967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] PATH_INFO: `/v3/projects/a3498bff28ef4749b0b1444c29b58717` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:12.040582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3498bff28ef4749b0b1444c29b58717)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:12.042951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:12.051310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1103/2209] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:59:11 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:12.055429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2853a15-a5b1-4614-9ef4-3ce3b6be2655 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:12.077772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:12.094399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3498bff28ef4749b0b1444c29b58717', 'request_id': 'req-b89fa3f0-86ef-430c-af44-8eaf756777b9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:12.100878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3498bff28ef4749b0b1444c29b58717', 'request_id': 'req-b89fa3f0-86ef-430c-af44-8eaf756777b9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:12.109638 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3498bff28ef4749b0b1444c29b58717', 'request_id': 'req-b89fa3f0-86ef-430c-af44-8eaf756777b9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:12.121536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] The token cache is being invalidate because project a3498bff28ef4749b0b1444c29b58717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:59:12.121790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:12.122507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:12.124148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1107/2210] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:59:12 2022] DELETE /identity/v3/projects/a3498bff28ef4749b0b1444c29b58717 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:12.130406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2853a15-a5b1-4614-9ef4-3ce3b6be2655 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:12.202687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920960d-26ed-4e1c-bccc-88eb9dff33c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:12.204634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920960d-26ed-4e1c-bccc-88eb9dff33c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:12.204634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920960d-26ed-4e1c-bccc-88eb9dff33c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:12.204634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920960d-26ed-4e1c-bccc-88eb9dff33c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:12.204634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920960d-26ed-4e1c-bccc-88eb9dff33c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:12.206977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920960d-26ed-4e1c-bccc-88eb9dff33c4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:12.241858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1104/2211] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:59:12 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:59:12.245393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:12.246905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:12.262002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fa3f0-86ef-430c-af44-8eaf756777b9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:12.276027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebef0c-4759-491e-b75d-a0f4ae4bcdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:12.276871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebef0c-4759-491e-b75d-a0f4ae4bcdde admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:12.277228 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebef0c-4759-491e-b75d-a0f4ae4bcdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:12.277571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebef0c-4759-491e-b75d-a0f4ae4bcdde admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:12.278556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebef0c-4759-491e-b75d-a0f4ae4bcdde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:12.280490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebef0c-4759-491e-b75d-a0f4ae4bcdde admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:12.292954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1108/2212] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:59:12.297950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920960d-26ed-4e1c-bccc-88eb9dff33c4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:12.310300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920960d-26ed-4e1c-bccc-88eb9dff33c4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:12.319445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbb039-1259-4703-ba8c-186a581406ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:12.320413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbb039-1259-4703-ba8c-186a581406ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:12.320692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbb039-1259-4703-ba8c-186a581406ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:12.322879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbb039-1259-4703-ba8c-186a581406ca admin admin] PATH_INFO: `/v3/projects/fa26b72bb5054865830d51e7408fbb0a/users/2d8a89e06941427da94ae25b2f006130/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:12.351916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbb039-1259-4703-ba8c-186a581406ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa26b72bb5054865830d51e7408fbb0a, user_id=2d8a89e06941427da94ae25b2f006130, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:12.351916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbb039-1259-4703-ba8c-186a581406ca admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:12.371105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1105/2213] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:59:12 2022] PUT /identity/v3/projects/fa26b72bb5054865830d51e7408fbb0a/users/2d8a89e06941427da94ae25b2f006130/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:12.375300 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebef0c-4759-491e-b75d-a0f4ae4bcdde admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:12.385745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebef0c-4759-491e-b75d-a0f4ae4bcdde admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:12.414436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:12.451369 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e90b8-5c9c-44b1-904c-a93c9686a5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:12.452053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e90b8-5c9c-44b1-904c-a93c9686a5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:12.452361 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e90b8-5c9c-44b1-904c-a93c9686a5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:12.452640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e90b8-5c9c-44b1-904c-a93c9686a5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:12.453381 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e90b8-5c9c-44b1-904c-a93c9686a5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:12.455050 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e90b8-5c9c-44b1-904c-a93c9686a5d0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:12.464740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1109/2214] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:59:12.469027 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbb039-1259-4703-ba8c-186a581406ca admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:12.480526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbb039-1259-4703-ba8c-186a581406ca admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:12.485847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd5b38-b301-4b64-90b8-cb5d7115fe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:12.486820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd5b38-b301-4b64-90b8-cb5d7115fe7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:12.487448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd5b38-b301-4b64-90b8-cb5d7115fe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:12.488068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd5b38-b301-4b64-90b8-cb5d7115fe7a admin admin] PATH_INFO: `/v3/projects/fa26b72bb5054865830d51e7408fbb0a/users/2d8a89e06941427da94ae25b2f006130/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:12.490193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd5b38-b301-4b64-90b8-cb5d7115fe7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa26b72bb5054865830d51e7408fbb0a, user_id=2d8a89e06941427da94ae25b2f006130, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:12.492879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd5b38-b301-4b64-90b8-cb5d7115fe7a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:12.504345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1106/2215] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:59:12 2022] PUT /identity/v3/projects/fa26b72bb5054865830d51e7408fbb0a/users/2d8a89e06941427da94ae25b2f006130/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:59:12.516328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3726eae-ba17-464a-a6be-7182778ae55e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:12.516585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3726eae-ba17-464a-a6be-7182778ae55e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:12.516893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3726eae-ba17-464a-a6be-7182778ae55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:12.549569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b3726eae-ba17-464a-a6be-7182778ae55e None None] MFA Rules not processed for user `2d8a89e06941427da94ae25b2f006130`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:12.550677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b3726eae-ba17-464a-a6be-7182778ae55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:12.629427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1110/2216] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:12.640869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd5b38-b301-4b64-90b8-cb5d7115fe7a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:12.642595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fefd5b38-b301-4b64-90b8-cb5d7115fe7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:12.713919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd5b38-b301-4b64-90b8-cb5d7115fe7a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:12.718259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf27491-9691-4722-8e9b-ec405d46ac31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:12.718917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf27491-9691-4722-8e9b-ec405d46ac31 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:12.719197 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf27491-9691-4722-8e9b-ec405d46ac31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:12.719466 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf27491-9691-4722-8e9b-ec405d46ac31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:12.729874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf27491-9691-4722-8e9b-ec405d46ac31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:12.731361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf27491-9691-4722-8e9b-ec405d46ac31 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:12.741140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1107/2217] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:59:12.877299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3726eae-ba17-464a-a6be-7182778ae55e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:12.940794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3726eae-ba17-464a-a6be-7182778ae55e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:12.946352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:12.947227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:12.947507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:12.947777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] PATH_INFO: `/v3/users/e32254f1275a4877af748c721ead0636` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:12.949089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32254f1275a4877af748c721ead0636)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:12.951106 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:12.999193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32254f1275a4877af748c721ead0636'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.001185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:13.008136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32254f1275a4877af748c721ead0636'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.008893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32254f1275a4877af748c721ead0636'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.017863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32254f1275a4877af748c721ead0636'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.029965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32254f1275a4877af748c721ead0636'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.036016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1111/2218] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:59:12 2022] DELETE /identity/v3/users/e32254f1275a4877af748c721ead0636 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:13.040622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887d227-7c4e-4b3d-bea2-1c1d7a85e50b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:13.040910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887d227-7c4e-4b3d-bea2-1c1d7a85e50b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:13.041183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887d227-7c4e-4b3d-bea2-1c1d7a85e50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:13.059852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8887d227-7c4e-4b3d-bea2-1c1d7a85e50b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:13.061088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8887d227-7c4e-4b3d-bea2-1c1d7a85e50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:13.074849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:13.128810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1108/2219] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 20:59:13.138803 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:13.140350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:13.197994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f20d-75d8-41f8-84c0-4edea7bc92ef admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:13.201889 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4410305-dc08-4971-99f7-9b0d8942e72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:13.202538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4410305-dc08-4971-99f7-9b0d8942e72f service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:13.202810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4410305-dc08-4971-99f7-9b0d8942e72f service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:13.203081 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4410305-dc08-4971-99f7-9b0d8942e72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:13.212547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4410305-dc08-4971-99f7-9b0d8942e72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:13.213974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4410305-dc08-4971-99f7-9b0d8942e72f service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:13.223733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1112/2220] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:59:13.548695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887d227-7c4e-4b3d-bea2-1c1d7a85e50b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:13.550284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8887d227-7c4e-4b3d-bea2-1c1d7a85e50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:13.563786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887d227-7c4e-4b3d-bea2-1c1d7a85e50b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:13.580030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:13.581032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:13.581428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:13.581766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] PATH_INFO: `/v3/projects/cabe41bcf26c41eb8b32910445415ea0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:13.583142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cabe41bcf26c41eb8b32910445415ea0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:13.585268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:13.635199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cabe41bcf26c41eb8b32910445415ea0', 'request_id': 'req-112a716f-3642-4ed0-983e-244d338359fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.641790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cabe41bcf26c41eb8b32910445415ea0', 'request_id': 'req-112a716f-3642-4ed0-983e-244d338359fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.650255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cabe41bcf26c41eb8b32910445415ea0', 'request_id': 'req-112a716f-3642-4ed0-983e-244d338359fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.661619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] The token cache is being invalidate because project cabe41bcf26c41eb8b32910445415ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:59:13.661907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.662542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.664198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1109/2221] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:59:13 2022] DELETE /identity/v3/projects/cabe41bcf26c41eb8b32910445415ea0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:59:13.668201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4410305-dc08-4971-99f7-9b0d8942e72f service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:13.678198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4410305-dc08-4971-99f7-9b0d8942e72f service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:13.742817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:13.743765 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:13.744050 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:13.744354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] PATH_INFO: `/v3/users/2d8a89e06941427da94ae25b2f006130` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:13.745558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d8a89e06941427da94ae25b2f006130)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:13.747614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:13.798160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d8a89e06941427da94ae25b2f006130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.805467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d8a89e06941427da94ae25b2f006130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.806505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d8a89e06941427da94ae25b2f006130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.820585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d8a89e06941427da94ae25b2f006130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.834877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:13.843872 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6dc85709-6055-4ac7-aae4-f82a53e540e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d8a89e06941427da94ae25b2f006130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:13.850980 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1113/2222] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:59:13 2022] DELETE /identity/v3/users/2d8a89e06941427da94ae25b2f006130 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:14.141712 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:14.143907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:14.159809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a716f-3642-4ed0-983e-244d338359fd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:14.217429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:14.218471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.218756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.219052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] PATH_INFO: `/v3/projects/fa26b72bb5054865830d51e7408fbb0a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.220295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa26b72bb5054865830d51e7408fbb0a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:14.222214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:14.267784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:14.270717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa26b72bb5054865830d51e7408fbb0a', 'request_id': 'req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:14.278506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa26b72bb5054865830d51e7408fbb0a', 'request_id': 'req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:14.286927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa26b72bb5054865830d51e7408fbb0a', 'request_id': 'req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:14.297360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] The token cache is being invalidate because project fa26b72bb5054865830d51e7408fbb0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:59:14.297659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:14.298270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca8a3f54-1c59-4600-ac6a-bd253a2dfc18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:14.299815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1110/2223] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:59:14 2022] DELETE /identity/v3/projects/fa26b72bb5054865830d51e7408fbb0a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:14.304816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9302f-b145-4ce2-9a87-e800ddb91609 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.305317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9302f-b145-4ce2-9a87-e800ddb91609 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.305423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9302f-b145-4ce2-9a87-e800ddb91609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.324446 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-84e9302f-b145-4ce2-9a87-e800ddb91609 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:14.326031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-84e9302f-b145-4ce2-9a87-e800ddb91609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:14.392865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1114/2224] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:14.400648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116f3c3-3615-4ae5-a371-5a7fe0e1b3db None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.400953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116f3c3-3615-4ae5-a371-5a7fe0e1b3db None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.401210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116f3c3-3615-4ae5-a371-5a7fe0e1b3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.420730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5116f3c3-3615-4ae5-a371-5a7fe0e1b3db None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:14.421932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5116f3c3-3615-4ae5-a371-5a7fe0e1b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:14.430462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1111/2225] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:14.435259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9302f-b145-4ce2-9a87-e800ddb91609 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:14.445181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9302f-b145-4ce2-9a87-e800ddb91609 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:14.450445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06979f68-2ca9-47c7-9029-95373a9f8fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:14.451099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06979f68-2ca9-47c7-9029-95373a9f8fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.451379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06979f68-2ca9-47c7-9029-95373a9f8fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.451650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06979f68-2ca9-47c7-9029-95373a9f8fc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.452413 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06979f68-2ca9-47c7-9029-95373a9f8fc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:59:14.452712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06979f68-2ca9-47c7-9029-95373a9f8fc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:14.454326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06979f68-2ca9-47c7-9029-95373a9f8fc4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:14.466079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1115/2226] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:59:14.470632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116f3c3-3615-4ae5-a371-5a7fe0e1b3db None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:14.480251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116f3c3-3615-4ae5-a371-5a7fe0e1b3db None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:14.486605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d43dc-c7a3-429c-87a8-604131873548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:14.487283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d43dc-c7a3-429c-87a8-604131873548 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.487537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d43dc-c7a3-429c-87a8-604131873548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.487808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d43dc-c7a3-429c-87a8-604131873548 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.488540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d43dc-c7a3-429c-87a8-604131873548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:14.490458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d43dc-c7a3-429c-87a8-604131873548 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:14.521988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1112/2227] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:59:14 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:14.523187 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06979f68-2ca9-47c7-9029-95373a9f8fc4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:14.535552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06979f68-2ca9-47c7-9029-95373a9f8fc4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:14.610128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41eb841-4bfd-4237-a38d-257a6005895d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:14.611122 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41eb841-4bfd-4237-a38d-257a6005895d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.611410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41eb841-4bfd-4237-a38d-257a6005895d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.611755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41eb841-4bfd-4237-a38d-257a6005895d admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.612871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41eb841-4bfd-4237-a38d-257a6005895d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:14.614840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41eb841-4bfd-4237-a38d-257a6005895d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:14.639138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1116/2228] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:59:14 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:14.643385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d43dc-c7a3-429c-87a8-604131873548 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:14.654595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d43dc-c7a3-429c-87a8-604131873548 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:14.660612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3595cb-7ddb-4873-a5a6-bf319a4b1782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:14.661400 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3595cb-7ddb-4873-a5a6-bf319a4b1782 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.661807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3595cb-7ddb-4873-a5a6-bf319a4b1782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.662198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3595cb-7ddb-4873-a5a6-bf319a4b1782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.663043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3595cb-7ddb-4873-a5a6-bf319a4b1782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:14.665305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3595cb-7ddb-4873-a5a6-bf319a4b1782 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:14.677851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1113/2229] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:59:14.682387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41eb841-4bfd-4237-a38d-257a6005895d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:14.693167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41eb841-4bfd-4237-a38d-257a6005895d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:14.697907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bf59a-019c-4c76-9397-73e166e3166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:14.698580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf59a-019c-4c76-9397-73e166e3166f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.698858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf59a-019c-4c76-9397-73e166e3166f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.699129 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf59a-019c-4c76-9397-73e166e3166f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.699862 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bf59a-019c-4c76-9397-73e166e3166f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:14.701776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bf59a-019c-4c76-9397-73e166e3166f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:14.715648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1117/2230] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:59:14.716891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3595cb-7ddb-4873-a5a6-bf319a4b1782 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:14.727122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3595cb-7ddb-4873-a5a6-bf319a4b1782 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:14.731638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a54908-ed6e-40fb-8e27-6ad130547231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:14.732572 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a54908-ed6e-40fb-8e27-6ad130547231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.732852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a54908-ed6e-40fb-8e27-6ad130547231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.733127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a54908-ed6e-40fb-8e27-6ad130547231 admin admin] PATH_INFO: `/v3/projects/5d3fea4c0dda4e2184bee1098aec9e4b/users/43fcec32e347431d91a70fc6df85905b/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.750011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a54908-ed6e-40fb-8e27-6ad130547231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d3fea4c0dda4e2184bee1098aec9e4b, user_id=43fcec32e347431d91a70fc6df85905b, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:14.752760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a54908-ed6e-40fb-8e27-6ad130547231 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:14.770341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1114/2231] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:59:14 2022] PUT /identity/v3/projects/5d3fea4c0dda4e2184bee1098aec9e4b/users/43fcec32e347431d91a70fc6df85905b/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:14.781086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00034c99-c739-4c41-ac14-06c40c293b82 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.781660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00034c99-c739-4c41-ac14-06c40c293b82 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.782183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00034c99-c739-4c41-ac14-06c40c293b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.812310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-00034c99-c739-4c41-ac14-06c40c293b82 None None] MFA Rules not processed for user `43fcec32e347431d91a70fc6df85905b`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:14.813519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-00034c99-c739-4c41-ac14-06c40c293b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:14.868387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:14.892064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1118/2232] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:14.896808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a54908-ed6e-40fb-8e27-6ad130547231 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:14.968027 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a54908-ed6e-40fb-8e27-6ad130547231 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:14.973098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ba209-bed5-48c1-89e4-86c6b1edcf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:14.973809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ba209-bed5-48c1-89e4-86c6b1edcf8b admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:14.974092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ba209-bed5-48c1-89e4-86c6b1edcf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:14.974364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ba209-bed5-48c1-89e4-86c6b1edcf8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:14.975066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ba209-bed5-48c1-89e4-86c6b1edcf8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:14.977029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ba209-bed5-48c1-89e4-86c6b1edcf8b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:15.006542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1115/2233] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:59:14 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:15.010713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00034c99-c739-4c41-ac14-06c40c293b82 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:15.022489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00034c99-c739-4c41-ac14-06c40c293b82 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:15.082710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fcfa1-50e6-4db5-b801-ef540244dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:15.083510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fcfa1-50e6-4db5-b801-ef540244dd25 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:15.083776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fcfa1-50e6-4db5-b801-ef540244dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:15.084054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fcfa1-50e6-4db5-b801-ef540244dd25 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:15.084787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fcfa1-50e6-4db5-b801-ef540244dd25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:15.086763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fcfa1-50e6-4db5-b801-ef540244dd25 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:15.108863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1119/2234] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:59:15 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:15.112758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ba209-bed5-48c1-89e4-86c6b1edcf8b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:15.124319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ba209-bed5-48c1-89e4-86c6b1edcf8b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:15.129791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3e449-dc7d-46b0-858a-2c06a947e88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:15.130954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3e449-dc7d-46b0-858a-2c06a947e88f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:15.131290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3e449-dc7d-46b0-858a-2c06a947e88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:15.131579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3e449-dc7d-46b0-858a-2c06a947e88f admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:15.132446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3e449-dc7d-46b0-858a-2c06a947e88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:15.134293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3e449-dc7d-46b0-858a-2c06a947e88f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:15.145703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1116/2235] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:59:15.149957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fcfa1-50e6-4db5-b801-ef540244dd25 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:15.160751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fcfa1-50e6-4db5-b801-ef540244dd25 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:15.167033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eebc7c-999c-4f0c-b84c-3b5b76a85f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:15.167961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eebc7c-999c-4f0c-b84c-3b5b76a85f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:15.168283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eebc7c-999c-4f0c-b84c-3b5b76a85f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:15.168586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eebc7c-999c-4f0c-b84c-3b5b76a85f47 admin admin] PATH_INFO: `/v3/projects/29ea80327f3f44a78e4e99493a6b9c29/users/a72bcae1b72c43d1980e444c9dda6395/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:15.186637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eebc7c-999c-4f0c-b84c-3b5b76a85f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ea80327f3f44a78e4e99493a6b9c29, user_id=a72bcae1b72c43d1980e444c9dda6395, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:15.189703 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eebc7c-999c-4f0c-b84c-3b5b76a85f47 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:15.203497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1120/2236] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:59:15 2022] PUT /identity/v3/projects/29ea80327f3f44a78e4e99493a6b9c29/users/a72bcae1b72c43d1980e444c9dda6395/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:59:15.208129 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3e449-dc7d-46b0-858a-2c06a947e88f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:15.228923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3e449-dc7d-46b0-858a-2c06a947e88f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:15.283457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:15.301390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a9bcd-e8dc-4879-b1a6-6ce6502d7258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:15.302078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a9bcd-e8dc-4879-b1a6-6ce6502d7258 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:15.302354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a9bcd-e8dc-4879-b1a6-6ce6502d7258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:15.302621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a9bcd-e8dc-4879-b1a6-6ce6502d7258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:15.303429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a9bcd-e8dc-4879-b1a6-6ce6502d7258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:15.305298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a9bcd-e8dc-4879-b1a6-6ce6502d7258 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:15.316150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1117/2237] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:59:15.320529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eebc7c-999c-4f0c-b84c-3b5b76a85f47 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:15.331323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eebc7c-999c-4f0c-b84c-3b5b76a85f47 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:15.336147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc318c8-2930-4542-a5ca-2d1687917544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:15.337050 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc318c8-2930-4542-a5ca-2d1687917544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:15.337322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc318c8-2930-4542-a5ca-2d1687917544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:15.337603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc318c8-2930-4542-a5ca-2d1687917544 admin admin] PATH_INFO: `/v3/projects/29ea80327f3f44a78e4e99493a6b9c29/users/a72bcae1b72c43d1980e444c9dda6395/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:15.339637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc318c8-2930-4542-a5ca-2d1687917544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ea80327f3f44a78e4e99493a6b9c29, user_id=a72bcae1b72c43d1980e444c9dda6395, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:15.342840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc318c8-2930-4542-a5ca-2d1687917544 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:15.353388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1121/2238] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:59:15 2022] PUT /identity/v3/projects/29ea80327f3f44a78e4e99493a6b9c29/users/a72bcae1b72c43d1980e444c9dda6395/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:59:15.364526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55350ec-b978-4cc1-82e8-2119bbc67ed8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:15.364828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55350ec-b978-4cc1-82e8-2119bbc67ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:15.365093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55350ec-b978-4cc1-82e8-2119bbc67ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:15.393833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d55350ec-b978-4cc1-82e8-2119bbc67ed8 None None] MFA Rules not processed for user `a72bcae1b72c43d1980e444c9dda6395`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:15.395637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d55350ec-b978-4cc1-82e8-2119bbc67ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:15.471099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1118/2239] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:15.482479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc318c8-2930-4542-a5ca-2d1687917544 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:15.484326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8cc318c8-2930-4542-a5ca-2d1687917544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:15.545432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc318c8-2930-4542-a5ca-2d1687917544 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:15.549343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb2320-3870-4d24-920c-a3e1ae7eaf3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:15.549981 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb2320-3870-4d24-920c-a3e1ae7eaf3e service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:15.550266 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb2320-3870-4d24-920c-a3e1ae7eaf3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:15.550542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb2320-3870-4d24-920c-a3e1ae7eaf3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:15.560664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb2320-3870-4d24-920c-a3e1ae7eaf3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:15.562110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb2320-3870-4d24-920c-a3e1ae7eaf3e service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:15.571637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1122/2240] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:59:16.201938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55350ec-b978-4cc1-82e8-2119bbc67ed8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:16.268071 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55350ec-b978-4cc1-82e8-2119bbc67ed8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:16.273003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:16.273893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:16.274174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:16.274570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] PATH_INFO: `/v3/users/43fcec32e347431d91a70fc6df85905b` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:16.275727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=43fcec32e347431d91a70fc6df85905b)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:16.277854 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:16.329433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43fcec32e347431d91a70fc6df85905b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:16.330137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43fcec32e347431d91a70fc6df85905b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:16.337296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43fcec32e347431d91a70fc6df85905b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:16.337934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43fcec32e347431d91a70fc6df85905b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:16.354362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:16.355355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43fcec32e347431d91a70fc6df85905b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:16.375488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1119/2241] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:59:16 2022] DELETE /identity/v3/users/43fcec32e347431d91a70fc6df85905b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:16.380100 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793f0c1-72dc-4d78-a7d8-275371ea478b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:16.380585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793f0c1-72dc-4d78-a7d8-275371ea478b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:16.380663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793f0c1-72dc-4d78-a7d8-275371ea478b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:16.401211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9793f0c1-72dc-4d78-a7d8-275371ea478b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:16.403737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9793f0c1-72dc-4d78-a7d8-275371ea478b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:16.451362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:16.486634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1123/2242] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:16.498388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:16.500776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:16.575755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7e8eb-da2d-44e3-b3a4-274e9be28cfe admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:16.580615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23960d2-0ed6-497a-8a67-b55c8436c0d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:16.581929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23960d2-0ed6-497a-8a67-b55c8436c0d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:16.582217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23960d2-0ed6-497a-8a67-b55c8436c0d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:16.583122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23960d2-0ed6-497a-8a67-b55c8436c0d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:16.594671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23960d2-0ed6-497a-8a67-b55c8436c0d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:16.596185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23960d2-0ed6-497a-8a67-b55c8436c0d8 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:16.606638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1120/2243] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 20:59:17.108526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793f0c1-72dc-4d78-a7d8-275371ea478b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:17.110188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9793f0c1-72dc-4d78-a7d8-275371ea478b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:17.130101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793f0c1-72dc-4d78-a7d8-275371ea478b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:17.136582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:17.137484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:17.137816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:17.138119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] PATH_INFO: `/v3/projects/5d3fea4c0dda4e2184bee1098aec9e4b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:17.139377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d3fea4c0dda4e2184bee1098aec9e4b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:17.141377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:17.193875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d3fea4c0dda4e2184bee1098aec9e4b', 'request_id': 'req-c38f8b05-38f6-408c-b2a9-4bf915ed8717'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.201170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d3fea4c0dda4e2184bee1098aec9e4b', 'request_id': 'req-c38f8b05-38f6-408c-b2a9-4bf915ed8717'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.218325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d3fea4c0dda4e2184bee1098aec9e4b', 'request_id': 'req-c38f8b05-38f6-408c-b2a9-4bf915ed8717'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.230978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] The token cache is being invalidate because project 5d3fea4c0dda4e2184bee1098aec9e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:59:17.231292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.232373 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.234058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1124/2244] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:59:17 2022] DELETE /identity/v3/projects/5d3fea4c0dda4e2184bee1098aec9e4b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:17.238098 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23960d2-0ed6-497a-8a67-b55c8436c0d8 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:17.248485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23960d2-0ed6-497a-8a67-b55c8436c0d8 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:17.310598 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:17.311735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:17.312025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:17.312401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] PATH_INFO: `/v3/users/a72bcae1b72c43d1980e444c9dda6395` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:17.313787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a72bcae1b72c43d1980e444c9dda6395)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:17.316019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:17.369479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a72bcae1b72c43d1980e444c9dda6395'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.370150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a72bcae1b72c43d1980e444c9dda6395'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.377616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a72bcae1b72c43d1980e444c9dda6395'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.378267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a72bcae1b72c43d1980e444c9dda6395'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.390962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-672d6f1e-8c92-47e2-9613-8ce0666e92a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a72bcae1b72c43d1980e444c9dda6395'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.405154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1121/2245] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:59:17 2022] DELETE /identity/v3/users/a72bcae1b72c43d1980e444c9dda6395 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:59:17.711791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:17.713996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:17.756898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f8b05-38f6-408c-b2a9-4bf915ed8717 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:17.823031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:17.824130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:17.824519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:17.829932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] PATH_INFO: `/v3/projects/29ea80327f3f44a78e4e99493a6b9c29` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:17.831762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ea80327f3f44a78e4e99493a6b9c29)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:17.837057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:17.896090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ea80327f3f44a78e4e99493a6b9c29', 'request_id': 'req-d5572794-f076-4ada-940b-d2ae0bcf10ac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.902879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ea80327f3f44a78e4e99493a6b9c29', 'request_id': 'req-d5572794-f076-4ada-940b-d2ae0bcf10ac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.912422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ea80327f3f44a78e4e99493a6b9c29', 'request_id': 'req-d5572794-f076-4ada-940b-d2ae0bcf10ac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.916852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:17.925940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] The token cache is being invalidate because project 29ea80327f3f44a78e4e99493a6b9c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:59:17.926246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.926934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d5572794-f076-4ada-940b-d2ae0bcf10ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:17.928602 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1125/2246] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:59:17 2022] DELETE /identity/v3/projects/29ea80327f3f44a78e4e99493a6b9c29 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:17.933890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ae39e-6dda-42d2-8ce3-5b953f6b23a1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:17.934169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ae39e-6dda-42d2-8ce3-5b953f6b23a1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:17.934523 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ae39e-6dda-42d2-8ce3-5b953f6b23a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:17.947216 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:17.954009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d96ae39e-6dda-42d2-8ce3-5b953f6b23a1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:17.956307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d96ae39e-6dda-42d2-8ce3-5b953f6b23a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:18.016637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1122/2247] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:18.024311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f4c7-fd72-4455-8ddd-beef70d3564b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.024598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f4c7-fd72-4455-8ddd-beef70d3564b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.025000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f4c7-fd72-4455-8ddd-beef70d3564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.046765 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c599f4c7-fd72-4455-8ddd-beef70d3564b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:18.047946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c599f4c7-fd72-4455-8ddd-beef70d3564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:18.056608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1126/2248] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:18.060924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ae39e-6dda-42d2-8ce3-5b953f6b23a1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:18.070663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ae39e-6dda-42d2-8ce3-5b953f6b23a1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:18.075769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a725-262c-40d7-b591-1ee9a42a9f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:18.076453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a725-262c-40d7-b591-1ee9a42a9f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.076732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a725-262c-40d7-b591-1ee9a42a9f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.077002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a725-262c-40d7-b591-1ee9a42a9f11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.077736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4a725-262c-40d7-b591-1ee9a42a9f11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:59:18.078039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4a725-262c-40d7-b591-1ee9a42a9f11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:18.079644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4a725-262c-40d7-b591-1ee9a42a9f11 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:18.091405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1123/2249] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:59:18.095518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599f4c7-fd72-4455-8ddd-beef70d3564b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:18.105308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599f4c7-fd72-4455-8ddd-beef70d3564b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:18.110362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc3ad2-5957-4930-91a9-13a50e5b0109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:18.110994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc3ad2-5957-4930-91a9-13a50e5b0109 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.111272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc3ad2-5957-4930-91a9-13a50e5b0109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.111535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc3ad2-5957-4930-91a9-13a50e5b0109 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.112275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc3ad2-5957-4930-91a9-13a50e5b0109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:18.114226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc3ad2-5957-4930-91a9-13a50e5b0109 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:18.142109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1127/2250] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:59:18 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:18.146137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a725-262c-40d7-b591-1ee9a42a9f11 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:18.157094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a725-262c-40d7-b591-1ee9a42a9f11 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:18.215141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a4d30-1097-4d7b-a322-1f7a377569f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:18.215862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a4d30-1097-4d7b-a322-1f7a377569f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.216166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a4d30-1097-4d7b-a322-1f7a377569f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.216424 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a4d30-1097-4d7b-a322-1f7a377569f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.217109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a4d30-1097-4d7b-a322-1f7a377569f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:18.219234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a4d30-1097-4d7b-a322-1f7a377569f3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:18.239651 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1124/2251] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:59:18 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:18.243699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc3ad2-5957-4930-91a9-13a50e5b0109 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:18.254586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc3ad2-5957-4930-91a9-13a50e5b0109 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:18.260355 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fabdaa-a89d-4fd6-92b5-e592fec5aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:18.261025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fabdaa-a89d-4fd6-92b5-e592fec5aa24 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.261303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fabdaa-a89d-4fd6-92b5-e592fec5aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.261574 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fabdaa-a89d-4fd6-92b5-e592fec5aa24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.262301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fabdaa-a89d-4fd6-92b5-e592fec5aa24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:18.264718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fabdaa-a89d-4fd6-92b5-e592fec5aa24 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:18.282935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1128/2252] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:59:18.283064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a4d30-1097-4d7b-a322-1f7a377569f3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:18.291141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a4d30-1097-4d7b-a322-1f7a377569f3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:18.296423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbba2d-282a-4e56-9410-7992d8bf430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:18.297245 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbba2d-282a-4e56-9410-7992d8bf430a admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.297590 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbba2d-282a-4e56-9410-7992d8bf430a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.297925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbba2d-282a-4e56-9410-7992d8bf430a admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.298752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbba2d-282a-4e56-9410-7992d8bf430a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:18.300481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbba2d-282a-4e56-9410-7992d8bf430a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:18.311225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1125/2253] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:59:18.315559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fabdaa-a89d-4fd6-92b5-e592fec5aa24 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:18.327839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fabdaa-a89d-4fd6-92b5-e592fec5aa24 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:18.333468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c60f5-0b35-4eb0-957e-42f6f984f461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:18.334432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c60f5-0b35-4eb0-957e-42f6f984f461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.334715 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c60f5-0b35-4eb0-957e-42f6f984f461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.335148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c60f5-0b35-4eb0-957e-42f6f984f461 admin admin] PATH_INFO: `/v3/projects/ab4e451a745c4200ac2665f74f9f7dd4/users/b94eebd253c74ae786498cb0fc4475ef/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.350492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c60f5-0b35-4eb0-957e-42f6f984f461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4e451a745c4200ac2665f74f9f7dd4, user_id=b94eebd253c74ae786498cb0fc4475ef, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:18.353330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c60f5-0b35-4eb0-957e-42f6f984f461 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:18.367142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1129/2254] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:59:18 2022] PUT /identity/v3/projects/ab4e451a745c4200ac2665f74f9f7dd4/users/b94eebd253c74ae786498cb0fc4475ef/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:18.377481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fc3a6-919c-48ec-ac78-f61c4637192a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.377771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fc3a6-919c-48ec-ac78-f61c4637192a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.378045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fc3a6-919c-48ec-ac78-f61c4637192a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.406648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-739fc3a6-919c-48ec-ac78-f61c4637192a None None] MFA Rules not processed for user `b94eebd253c74ae786498cb0fc4475ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:18.407833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-739fc3a6-919c-48ec-ac78-f61c4637192a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:18.482823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1126/2255] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:18.496575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c60f5-0b35-4eb0-957e-42f6f984f461 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:18.498847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-035c60f5-0b35-4eb0-957e-42f6f984f461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:18.563927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c60f5-0b35-4eb0-957e-42f6f984f461 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:18.567558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624779c7-62ac-452e-b836-43dc4127e651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:18.568246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624779c7-62ac-452e-b836-43dc4127e651 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.568529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624779c7-62ac-452e-b836-43dc4127e651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.568805 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624779c7-62ac-452e-b836-43dc4127e651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.577940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624779c7-62ac-452e-b836-43dc4127e651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:18.579751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624779c7-62ac-452e-b836-43dc4127e651 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:18.589796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1130/2256] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:59:18.613508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fc3a6-919c-48ec-ac78-f61c4637192a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:18.618535 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:18.683838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fc3a6-919c-48ec-ac78-f61c4637192a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:18.689358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:18.690256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.690534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.690806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] PATH_INFO: `/v3/users/b94eebd253c74ae786498cb0fc4475ef` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.692236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94eebd253c74ae786498cb0fc4475ef)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:18.694223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:18.745452 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94eebd253c74ae786498cb0fc4475ef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:18.746118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94eebd253c74ae786498cb0fc4475ef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:18.753412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94eebd253c74ae786498cb0fc4475ef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:18.754052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94eebd253c74ae786498cb0fc4475ef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:18.766181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94eebd253c74ae786498cb0fc4475ef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:18.779675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1127/2257] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:59:18 2022] DELETE /identity/v3/users/b94eebd253c74ae786498cb0fc4475ef => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:18.784246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8f034-891f-4559-93e9-15d03006af78 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.784748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8f034-891f-4559-93e9-15d03006af78 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.784886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8f034-891f-4559-93e9-15d03006af78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.802339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a9d8f034-891f-4559-93e9-15d03006af78 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:18.803540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a9d8f034-891f-4559-93e9-15d03006af78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:18.871468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1131/2258] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:18.881958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:18.883429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:18.955701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5ee37-0f8f-41be-9153-d525c7356741 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:18.959382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640d64a-3d3f-470c-bda6-272fbcf96dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:18.960094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640d64a-3d3f-470c-bda6-272fbcf96dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:18.960397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640d64a-3d3f-470c-bda6-272fbcf96dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:18.960667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640d64a-3d3f-470c-bda6-272fbcf96dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:18.971080 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640d64a-3d3f-470c-bda6-272fbcf96dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:18.972555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640d64a-3d3f-470c-bda6-272fbcf96dc3 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:18.983970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1128/2259] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:59:19.311281 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8f034-891f-4559-93e9-15d03006af78 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:19.312880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a9d8f034-891f-4559-93e9-15d03006af78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:19.325433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8f034-891f-4559-93e9-15d03006af78 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:19.330673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:19.331570 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:19.331848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:19.332137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] PATH_INFO: `/v3/projects/ab4e451a745c4200ac2665f74f9f7dd4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:19.333380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab4e451a745c4200ac2665f74f9f7dd4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:19.335409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:19.371678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:19.388776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab4e451a745c4200ac2665f74f9f7dd4', 'request_id': 'req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:19.395174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab4e451a745c4200ac2665f74f9f7dd4', 'request_id': 'req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:19.404312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab4e451a745c4200ac2665f74f9f7dd4', 'request_id': 'req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:19.414697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] The token cache is being invalidate because project ab4e451a745c4200ac2665f74f9f7dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:59:19.415027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:19.415627 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1cdcf8d6-b0f9-44ca-9865-3f48657f3150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:19.417294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1132/2260] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:59:19 2022] DELETE /identity/v3/projects/ab4e451a745c4200ac2665f74f9f7dd4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:19.422214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca726142-09d6-4106-8f11-4214fefe8ef6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:19.422680 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca726142-09d6-4106-8f11-4214fefe8ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:19.422883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca726142-09d6-4106-8f11-4214fefe8ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:19.442022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ca726142-09d6-4106-8f11-4214fefe8ef6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:19.443268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ca726142-09d6-4106-8f11-4214fefe8ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:19.464261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:19.509953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1129/2261] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:19.519948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf0482-ec88-4a31-92eb-a3d36280c06b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:19.520270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf0482-ec88-4a31-92eb-a3d36280c06b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:19.520544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf0482-ec88-4a31-92eb-a3d36280c06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:19.542005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5dcf0482-ec88-4a31-92eb-a3d36280c06b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:19.543245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5dcf0482-ec88-4a31-92eb-a3d36280c06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:19.556662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1133/2262] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:19.562411 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca726142-09d6-4106-8f11-4214fefe8ef6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:19.572601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca726142-09d6-4106-8f11-4214fefe8ef6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:19.578274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc331be4-5305-4373-9c80-b2889446cf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:19.578956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc331be4-5305-4373-9c80-b2889446cf93 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:19.579286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc331be4-5305-4373-9c80-b2889446cf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:19.579559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc331be4-5305-4373-9c80-b2889446cf93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:19.580353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc331be4-5305-4373-9c80-b2889446cf93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:59:19.580659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc331be4-5305-4373-9c80-b2889446cf93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:19.582447 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc331be4-5305-4373-9c80-b2889446cf93 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:19.596844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1130/2263] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:59:19.601271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf0482-ec88-4a31-92eb-a3d36280c06b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:19.611229 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf0482-ec88-4a31-92eb-a3d36280c06b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:19.617878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55483af-9b0c-4463-81b4-32bf717e1fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:19.618678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55483af-9b0c-4463-81b4-32bf717e1fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:19.618901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55483af-9b0c-4463-81b4-32bf717e1fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:19.619167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55483af-9b0c-4463-81b4-32bf717e1fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:19.619874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55483af-9b0c-4463-81b4-32bf717e1fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:19.621859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55483af-9b0c-4463-81b4-32bf717e1fb0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:19.648763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1134/2264] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:59:19 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:19.652905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc331be4-5305-4373-9c80-b2889446cf93 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:19.663396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc331be4-5305-4373-9c80-b2889446cf93 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:19.735877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00325f4-2b5a-43b0-823f-a8c9a80ce333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:19.736687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00325f4-2b5a-43b0-823f-a8c9a80ce333 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:19.737028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00325f4-2b5a-43b0-823f-a8c9a80ce333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:19.737364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00325f4-2b5a-43b0-823f-a8c9a80ce333 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:19.738139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00325f4-2b5a-43b0-823f-a8c9a80ce333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:19.740199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00325f4-2b5a-43b0-823f-a8c9a80ce333 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:19.768335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1131/2265] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:59:19 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:59:19.772421 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55483af-9b0c-4463-81b4-32bf717e1fb0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:19.791582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55483af-9b0c-4463-81b4-32bf717e1fb0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:19.801389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91336d-efa8-4c89-9ec7-e1859d32a7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:19.802251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91336d-efa8-4c89-9ec7-e1859d32a7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:19.802745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91336d-efa8-4c89-9ec7-e1859d32a7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:19.803270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91336d-efa8-4c89-9ec7-e1859d32a7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:19.803965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91336d-efa8-4c89-9ec7-e1859d32a7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:19.807213 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91336d-efa8-4c89-9ec7-e1859d32a7c9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:19.821363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1135/2266] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:59:19.825416 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00325f4-2b5a-43b0-823f-a8c9a80ce333 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:19.839186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00325f4-2b5a-43b0-823f-a8c9a80ce333 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:19.844472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff975d5b-4616-417d-9f77-1245b4b3e8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:19.845147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff975d5b-4616-417d-9f77-1245b4b3e8ea admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:19.845425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff975d5b-4616-417d-9f77-1245b4b3e8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:19.845693 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff975d5b-4616-417d-9f77-1245b4b3e8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:19.846428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff975d5b-4616-417d-9f77-1245b4b3e8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:19.848035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff975d5b-4616-417d-9f77-1245b4b3e8ea admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:19.862043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1132/2267] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:59:19.863432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91336d-efa8-4c89-9ec7-e1859d32a7c9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:19.874218 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91336d-efa8-4c89-9ec7-e1859d32a7c9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:19.879372 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b9260-ef55-4fbe-a18c-4bfc39125ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:19.880314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b9260-ef55-4fbe-a18c-4bfc39125ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:19.880750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b9260-ef55-4fbe-a18c-4bfc39125ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:19.881025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b9260-ef55-4fbe-a18c-4bfc39125ff0 admin admin] PATH_INFO: `/v3/projects/52fd0e385e8540dc9bd584afa498d47c/users/9dd7b22d66d54c1dba59708a53146ff5/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:19.899312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b9260-ef55-4fbe-a18c-4bfc39125ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52fd0e385e8540dc9bd584afa498d47c, user_id=9dd7b22d66d54c1dba59708a53146ff5, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:19.903807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b9260-ef55-4fbe-a18c-4bfc39125ff0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:19.920391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1136/2268] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:59:19 2022] PUT /identity/v3/projects/52fd0e385e8540dc9bd584afa498d47c/users/9dd7b22d66d54c1dba59708a53146ff5/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:59:19.934308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4833a-4091-4181-8aa6-3a883145bcab None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:19.934708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4833a-4091-4181-8aa6-3a883145bcab None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:19.934924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4833a-4091-4181-8aa6-3a883145bcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:19.966995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7dd4833a-4091-4181-8aa6-3a883145bcab None None] MFA Rules not processed for user `9dd7b22d66d54c1dba59708a53146ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:19.968301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7dd4833a-4091-4181-8aa6-3a883145bcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:20.035389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1133/2269] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:20.046545 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b9260-ef55-4fbe-a18c-4bfc39125ff0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:20.048961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-162b9260-ef55-4fbe-a18c-4bfc39125ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:20.112759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b9260-ef55-4fbe-a18c-4bfc39125ff0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:20.116442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643acbeb-e9bf-482d-8cb5-79987ed15d4b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097b12fbf6494336b11ece5c498a4103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:20.117093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643acbeb-e9bf-482d-8cb5-79987ed15d4b service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:20.117434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643acbeb-e9bf-482d-8cb5-79987ed15d4b service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:20.117706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643acbeb-e9bf-482d-8cb5-79987ed15d4b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:20.128516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643acbeb-e9bf-482d-8cb5-79987ed15d4b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:20.130333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643acbeb-e9bf-482d-8cb5-79987ed15d4b service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:20.140618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1137/2270] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:59:24.791202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3180e-fc4d-4116-87fa-e0eca970b311 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:24.791819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3180e-fc4d-4116-87fa-e0eca970b311 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:24.791819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3180e-fc4d-4116-87fa-e0eca970b311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:24.812001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b2d3180e-fc4d-4116-87fa-e0eca970b311 None None] MFA Rules not processed for user `934b314c46824d5397a0ae92bb07b725`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:24.813407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b2d3180e-fc4d-4116-87fa-e0eca970b311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:24.832198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:24.881729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1134/2271] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:29.656756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643acbeb-e9bf-482d-8cb5-79987ed15d4b service glance] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:29.658909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-643acbeb-e9bf-482d-8cb5-79987ed15d4b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:29.771409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:29.780146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643acbeb-e9bf-482d-8cb5-79987ed15d4b service glance] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:29.784508 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:29.785709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:29.786166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:29.786834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] PATH_INFO: `/v3/users/87fe34dc828e4169aa5f7b7e876351ff` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:29.788611 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87fe34dc828e4169aa5f7b7e876351ff)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:29.791047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:29.859551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87fe34dc828e4169aa5f7b7e876351ff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:29.866921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87fe34dc828e4169aa5f7b7e876351ff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:29.868207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87fe34dc828e4169aa5f7b7e876351ff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:29.877756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87fe34dc828e4169aa5f7b7e876351ff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:29.896640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87fe34dc828e4169aa5f7b7e876351ff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:29.903610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1138/2272] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:59:29 2022] DELETE /identity/v3/users/87fe34dc828e4169aa5f7b7e876351ff => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:29.908181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ed5a6-a94d-42c6-adc3-7daf8f3ca1f7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:29.908831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ed5a6-a94d-42c6-adc3-7daf8f3ca1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:29.908831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ed5a6-a94d-42c6-adc3-7daf8f3ca1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:29.927214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-626ed5a6-a94d-42c6-adc3-7daf8f3ca1f7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:29.928589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-626ed5a6-a94d-42c6-adc3-7daf8f3ca1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:29.994208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1135/2273] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:30.004199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:30.006366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:30.066311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e845e-94de-4791-b598-f02f5194fe0d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:30.070332 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe39a6-f54f-4218-ae85-0db8d955231b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:30.071061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe39a6-f54f-4218-ae85-0db8d955231b service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:30.071359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe39a6-f54f-4218-ae85-0db8d955231b service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:30.071683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe39a6-f54f-4218-ae85-0db8d955231b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:30.083700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe39a6-f54f-4218-ae85-0db8d955231b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:30.085272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe39a6-f54f-4218-ae85-0db8d955231b service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:30.096647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1139/2274] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:59:30.408433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ed5a6-a94d-42c6-adc3-7daf8f3ca1f7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:30.410226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-626ed5a6-a94d-42c6-adc3-7daf8f3ca1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:30.426445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ed5a6-a94d-42c6-adc3-7daf8f3ca1f7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:30.432557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:30.433563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:30.433948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:30.434260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] PATH_INFO: `/v3/projects/55b6d41ed94b4765926d065244f1f2a8` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:30.436313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55b6d41ed94b4765926d065244f1f2a8)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:30.438274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:30.487880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55b6d41ed94b4765926d065244f1f2a8', 'request_id': 'req-dcdc7640-ec28-4e37-b970-82cab3658938'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:30.494218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55b6d41ed94b4765926d065244f1f2a8', 'request_id': 'req-dcdc7640-ec28-4e37-b970-82cab3658938'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:30.503686 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55b6d41ed94b4765926d065244f1f2a8', 'request_id': 'req-dcdc7640-ec28-4e37-b970-82cab3658938'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:30.513937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] The token cache is being invalidate because project 55b6d41ed94b4765926d065244f1f2a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:59:30.514222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:30.514881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcdc7640-ec28-4e37-b970-82cab3658938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:30.516536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1136/2275] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:59:30 2022] DELETE /identity/v3/projects/55b6d41ed94b4765926d065244f1f2a8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 20:59:30.521784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c051b-03cd-43ef-a1c5-9a2ca4717c6b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:30.522132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c051b-03cd-43ef-a1c5-9a2ca4717c6b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:30.522417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c051b-03cd-43ef-a1c5-9a2ca4717c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:30.541285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-976c051b-03cd-43ef-a1c5-9a2ca4717c6b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:30.542629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-976c051b-03cd-43ef-a1c5-9a2ca4717c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:30.609911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1140/2276] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:30.618558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0502496-a875-4363-a97f-773c9389ff56 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:30.618846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0502496-a875-4363-a97f-773c9389ff56 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:30.619119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0502496-a875-4363-a97f-773c9389ff56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:30.638883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b0502496-a875-4363-a97f-773c9389ff56 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:30.640206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b0502496-a875-4363-a97f-773c9389ff56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:30.652684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1137/2277] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:30.657385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c051b-03cd-43ef-a1c5-9a2ca4717c6b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:30.668693 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c051b-03cd-43ef-a1c5-9a2ca4717c6b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:30.674674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d0521-9961-481b-8ea7-14edaa4f664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:30.675263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d0521-9961-481b-8ea7-14edaa4f664b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:30.675535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d0521-9961-481b-8ea7-14edaa4f664b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:30.675813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d0521-9961-481b-8ea7-14edaa4f664b admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:30.676571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d0521-9961-481b-8ea7-14edaa4f664b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:59:30.676903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d0521-9961-481b-8ea7-14edaa4f664b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:30.678757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d0521-9961-481b-8ea7-14edaa4f664b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:30.692614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1141/2278] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:59:30.697714 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0502496-a875-4363-a97f-773c9389ff56 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:30.707115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0502496-a875-4363-a97f-773c9389ff56 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:30.712887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8545da-9f31-4a2e-b899-dc1bd0ba563b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:30.713536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8545da-9f31-4a2e-b899-dc1bd0ba563b admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:30.713839 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8545da-9f31-4a2e-b899-dc1bd0ba563b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:30.714111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8545da-9f31-4a2e-b899-dc1bd0ba563b admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:30.714806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8545da-9f31-4a2e-b899-dc1bd0ba563b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:30.716865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8545da-9f31-4a2e-b899-dc1bd0ba563b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:30.744688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1138/2279] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:59:30 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:30.749541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d0521-9961-481b-8ea7-14edaa4f664b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:30.756872 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:30.761436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d0521-9961-481b-8ea7-14edaa4f664b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:30.832730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15451de1-3767-4e6d-8dbc-fed1d8a9829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:30.832730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15451de1-3767-4e6d-8dbc-fed1d8a9829c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:30.832730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15451de1-3767-4e6d-8dbc-fed1d8a9829c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:30.832730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15451de1-3767-4e6d-8dbc-fed1d8a9829c admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:30.833279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15451de1-3767-4e6d-8dbc-fed1d8a9829c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:30.834355 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15451de1-3767-4e6d-8dbc-fed1d8a9829c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:30.860286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1142/2280] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:59:30 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 20:59:30.861568 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8545da-9f31-4a2e-b899-dc1bd0ba563b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:30.872720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8545da-9f31-4a2e-b899-dc1bd0ba563b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:30.878148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e9afb-e889-469e-99ff-a419a8fc8c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:30.878825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e9afb-e889-469e-99ff-a419a8fc8c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:30.879112 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e9afb-e889-469e-99ff-a419a8fc8c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:30.879389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e9afb-e889-469e-99ff-a419a8fc8c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:30.880288 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e9afb-e889-469e-99ff-a419a8fc8c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:30.887105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e9afb-e889-469e-99ff-a419a8fc8c68 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:30.894228 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1139/2281] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:59:30.899156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15451de1-3767-4e6d-8dbc-fed1d8a9829c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:30.909359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15451de1-3767-4e6d-8dbc-fed1d8a9829c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:30.914589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db058da6-f913-4794-90de-f496c716a6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:30.915331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db058da6-f913-4794-90de-f496c716a6ae admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:30.915506 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db058da6-f913-4794-90de-f496c716a6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:30.915803 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db058da6-f913-4794-90de-f496c716a6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:30.916581 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db058da6-f913-4794-90de-f496c716a6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:30.918192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db058da6-f913-4794-90de-f496c716a6ae admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:30.933038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1143/2282] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 20:59:30.933937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e9afb-e889-469e-99ff-a419a8fc8c68 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:30.945041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e9afb-e889-469e-99ff-a419a8fc8c68 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:30.950657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dd8cb-0ce8-4d52-b570-f9dc32a8b1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:30.951576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dd8cb-0ce8-4d52-b570-f9dc32a8b1f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:30.951835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dd8cb-0ce8-4d52-b570-f9dc32a8b1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:30.952136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dd8cb-0ce8-4d52-b570-f9dc32a8b1f4 admin admin] PATH_INFO: `/v3/projects/36d40e24cac74460bbbed4dffce9a4da/users/63ec8ab7842a443c922a39f5ffcfd51a/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:30.955803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:30.980528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dd8cb-0ce8-4d52-b570-f9dc32a8b1f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d40e24cac74460bbbed4dffce9a4da, user_id=63ec8ab7842a443c922a39f5ffcfd51a, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:30.983237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dd8cb-0ce8-4d52-b570-f9dc32a8b1f4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:30.996916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1140/2283] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:59:30 2022] PUT /identity/v3/projects/36d40e24cac74460bbbed4dffce9a4da/users/63ec8ab7842a443c922a39f5ffcfd51a/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:31.002041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c20d75-619a-477f-96bf-20596884e8fa None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:31.002330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c20d75-619a-477f-96bf-20596884e8fa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:31.002604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c20d75-619a-477f-96bf-20596884e8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:31.020827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-86c20d75-619a-477f-96bf-20596884e8fa None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:31.022032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-86c20d75-619a-477f-96bf-20596884e8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:31.084833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1144/2284] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:31.094846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dd8cb-0ce8-4d52-b570-f9dc32a8b1f4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:31.096857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-704dd8cb-0ce8-4d52-b570-f9dc32a8b1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:31.165312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dd8cb-0ce8-4d52-b570-f9dc32a8b1f4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:31.170327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf341d-02de-438c-bc05-6daf0889c3d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:31.171266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf341d-02de-438c-bc05-6daf0889c3d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:31.171620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf341d-02de-438c-bc05-6daf0889c3d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:31.171957 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf341d-02de-438c-bc05-6daf0889c3d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:31.184364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf341d-02de-438c-bc05-6daf0889c3d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:31.185862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf341d-02de-438c-bc05-6daf0889c3d2 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:31.196843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1141/2285] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:59:31.963494 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16fe07-b246-4f6c-958d-91eb51b35ef3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:31.963951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16fe07-b246-4f6c-958d-91eb51b35ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:31.964132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16fe07-b246-4f6c-958d-91eb51b35ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:32.013809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8a16fe07-b246-4f6c-958d-91eb51b35ef3 None None] MFA Rules not processed for user `63ec8ab7842a443c922a39f5ffcfd51a`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:32.015299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8a16fe07-b246-4f6c-958d-91eb51b35ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:32.121526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1145/2286] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:32.133043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf341d-02de-438c-bc05-6daf0889c3d2 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:32.136223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dccf341d-02de-438c-bc05-6daf0889c3d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:32.226592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf341d-02de-438c-bc05-6daf0889c3d2 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:32.232514 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc1c40-3204-4e38-aa9b-da822d128bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:32.233307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc1c40-3204-4e38-aa9b-da822d128bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:32.233653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc1c40-3204-4e38-aa9b-da822d128bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:32.233996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc1c40-3204-4e38-aa9b-da822d128bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:32.253124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc1c40-3204-4e38-aa9b-da822d128bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:32.253124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc1c40-3204-4e38-aa9b-da822d128bd0 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:32.259806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1142/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:59:34.683838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4db22d-c905-4aaa-92d7-2e40f582e042 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:34.684393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4db22d-c905-4aaa-92d7-2e40f582e042 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:34.684607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4db22d-c905-4aaa-92d7-2e40f582e042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:34.706037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9b4db22d-c905-4aaa-92d7-2e40f582e042 None None] MFA Rules not processed for user `63ec8ab7842a443c922a39f5ffcfd51a`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:34.707284 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9b4db22d-c905-4aaa-92d7-2e40f582e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:34.716258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1146/2288] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:42.254840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=58,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:44.699222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:44.753300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e8ac9-7907-4d66-b1a0-a753e7bfabda None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:44.753760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e8ac9-7907-4d66-b1a0-a753e7bfabda None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:44.753890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e8ac9-7907-4d66-b1a0-a753e7bfabda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:44.771504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-185e8ac9-7907-4d66-b1a0-a753e7bfabda None None] MFA Rules not processed for user `aa7e6c5842a84e91ac94c1107ad29158`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:44.772743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-185e8ac9-7907-4d66-b1a0-a753e7bfabda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:44.832701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1143/2289] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:49.525999 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4db22d-c905-4aaa-92d7-2e40f582e042 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:49.527655 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9b4db22d-c905-4aaa-92d7-2e40f582e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:49.541586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4db22d-c905-4aaa-92d7-2e40f582e042 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:49.546470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:49.547351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:49.547669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:49.547955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] PATH_INFO: `/v3/users/9dd7b22d66d54c1dba59708a53146ff5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:49.549206 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dd7b22d66d54c1dba59708a53146ff5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:49.551149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:49.599886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dd7b22d66d54c1dba59708a53146ff5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:49.606618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dd7b22d66d54c1dba59708a53146ff5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:49.607307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dd7b22d66d54c1dba59708a53146ff5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:49.614777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dd7b22d66d54c1dba59708a53146ff5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:49.627584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dd7b22d66d54c1dba59708a53146ff5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:49.633710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1147/2290] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 20:59:49 2022] DELETE /identity/v3/users/9dd7b22d66d54c1dba59708a53146ff5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:49.638309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d45cd7-c3f7-4f8b-8435-f86c20e09d64 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:49.638804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d45cd7-c3f7-4f8b-8435-f86c20e09d64 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:49.638902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d45cd7-c3f7-4f8b-8435-f86c20e09d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:49.656722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a9d45cd7-c3f7-4f8b-8435-f86c20e09d64 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:49.657927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a9d45cd7-c3f7-4f8b-8435-f86c20e09d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:49.727554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1144/2291] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:49.737511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:49.739068 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:49.796688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a46b46-c75c-40ee-b0ff-475764293fee admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:49.800290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bc04f-4dd6-41ca-80f4-75730aa26d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:49.800937 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bc04f-4dd6-41ca-80f4-75730aa26d34 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:49.801219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bc04f-4dd6-41ca-80f4-75730aa26d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:49.801507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bc04f-4dd6-41ca-80f4-75730aa26d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:49.810985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3bc04f-4dd6-41ca-80f4-75730aa26d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:49.812514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3bc04f-4dd6-41ca-80f4-75730aa26d34 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:49.823501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1148/2292] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 20:59:50.168940 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d45cd7-c3f7-4f8b-8435-f86c20e09d64 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:50.170694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a9d45cd7-c3f7-4f8b-8435-f86c20e09d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:50.186423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d45cd7-c3f7-4f8b-8435-f86c20e09d64 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:50.191604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:50.193017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:50.193367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:50.193646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] PATH_INFO: `/v3/projects/52fd0e385e8540dc9bd584afa498d47c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:50.195016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] RBAC: Authorizing `identity:delete_project(project_id=52fd0e385e8540dc9bd584afa498d47c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:50.197118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:50.248325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52fd0e385e8540dc9bd584afa498d47c', 'request_id': 'req-faa872d2-e526-4bf7-9721-8f0b86f0614c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:50.254731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52fd0e385e8540dc9bd584afa498d47c', 'request_id': 'req-faa872d2-e526-4bf7-9721-8f0b86f0614c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:50.264548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52fd0e385e8540dc9bd584afa498d47c', 'request_id': 'req-faa872d2-e526-4bf7-9721-8f0b86f0614c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:50.281508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] The token cache is being invalidate because project 52fd0e385e8540dc9bd584afa498d47c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 20:59:50.281914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:50.282978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa872d2-e526-4bf7-9721-8f0b86f0614c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 20:59:50.285444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1145/2293] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 20:59:50 2022] DELETE /identity/v3/projects/52fd0e385e8540dc9bd584afa498d47c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:50.290556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ec772-fd2e-418c-9c60-5bfee981b80c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:50.290850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ec772-fd2e-418c-9c60-5bfee981b80c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:50.291194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ec772-fd2e-418c-9c60-5bfee981b80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:50.311502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9e4ec772-fd2e-418c-9c60-5bfee981b80c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:50.312743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9e4ec772-fd2e-418c-9c60-5bfee981b80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:50.385498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1149/2294] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:50.394323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20678b15-ebb2-4d0b-903b-d1d5f5b9fb5b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:50.394798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20678b15-ebb2-4d0b-903b-d1d5f5b9fb5b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:50.395187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20678b15-ebb2-4d0b-903b-d1d5f5b9fb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:50.421097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-20678b15-ebb2-4d0b-903b-d1d5f5b9fb5b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:50.422876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-20678b15-ebb2-4d0b-903b-d1d5f5b9fb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:50.432453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1146/2295] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:50.437610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ec772-fd2e-418c-9c60-5bfee981b80c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:50.449474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ec772-fd2e-418c-9c60-5bfee981b80c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:50.457419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca028957-9729-46ae-9f8b-dcf9913efb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:50.459144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca028957-9729-46ae-9f8b-dcf9913efb8e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:50.459385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca028957-9729-46ae-9f8b-dcf9913efb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:50.459737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca028957-9729-46ae-9f8b-dcf9913efb8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:50.460919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca028957-9729-46ae-9f8b-dcf9913efb8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 20:59:50.461319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca028957-9729-46ae-9f8b-dcf9913efb8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:50.463633 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca028957-9729-46ae-9f8b-dcf9913efb8e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:50.477101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1150/2296] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 20:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 20:59:50.481820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20678b15-ebb2-4d0b-903b-d1d5f5b9fb5b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:50.492018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20678b15-ebb2-4d0b-903b-d1d5f5b9fb5b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:50.497974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2048e9-3462-42c9-a31c-a5a0902c387b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:50.498667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2048e9-3462-42c9-a31c-a5a0902c387b admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:50.498940 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2048e9-3462-42c9-a31c-a5a0902c387b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:50.499225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2048e9-3462-42c9-a31c-a5a0902c387b admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:50.499934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2048e9-3462-42c9-a31c-a5a0902c387b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:50.501921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2048e9-3462-42c9-a31c-a5a0902c387b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:50.528872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:50.537362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1147/2297] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 20:59:50 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:50.541514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca028957-9729-46ae-9f8b-dcf9913efb8e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:50.547829 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 20:59:50.554288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca028957-9729-46ae-9f8b-dcf9913efb8e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:50.620618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff9482-9160-4283-a6e3-4c6c6628727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:50.621386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff9482-9160-4283-a6e3-4c6c6628727c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:50.621672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff9482-9160-4283-a6e3-4c6c6628727c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:50.621949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff9482-9160-4283-a6e3-4c6c6628727c admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:50.622657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff9482-9160-4283-a6e3-4c6c6628727c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:50.624699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff9482-9160-4283-a6e3-4c6c6628727c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:50.650365 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1151/2298] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 20:59:50 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 20:59:50.651421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2048e9-3462-42c9-a31c-a5a0902c387b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:50.661891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2048e9-3462-42c9-a31c-a5a0902c387b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:50.667557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae2610-e9ba-4c85-adc4-3ed85c49fd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:50.668316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae2610-e9ba-4c85-adc4-3ed85c49fd26 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:50.669171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae2610-e9ba-4c85-adc4-3ed85c49fd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:50.669171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae2610-e9ba-4c85-adc4-3ed85c49fd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:50.669652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae2610-e9ba-4c85-adc4-3ed85c49fd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:50.671511 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae2610-e9ba-4c85-adc4-3ed85c49fd26 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:50.684886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1148/2299] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:59:50.688999 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff9482-9160-4283-a6e3-4c6c6628727c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:50.700518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff9482-9160-4283-a6e3-4c6c6628727c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:50.712132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa93f74-d7ca-4c1b-9e93-31b76b4e80d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:50.713288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa93f74-d7ca-4c1b-9e93-31b76b4e80d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:50.713780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa93f74-d7ca-4c1b-9e93-31b76b4e80d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:50.714096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa93f74-d7ca-4c1b-9e93-31b76b4e80d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:50.714926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa93f74-d7ca-4c1b-9e93-31b76b4e80d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:50.716791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa93f74-d7ca-4c1b-9e93-31b76b4e80d8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:50.729568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1152/2300] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 20:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 20:59:50.733790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae2610-e9ba-4c85-adc4-3ed85c49fd26 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:50.744166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae2610-e9ba-4c85-adc4-3ed85c49fd26 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:50.749622 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cad78-a16e-4e36-8091-4e99a97b934c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:50.750791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cad78-a16e-4e36-8091-4e99a97b934c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:50.751233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cad78-a16e-4e36-8091-4e99a97b934c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:50.751662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cad78-a16e-4e36-8091-4e99a97b934c admin admin] PATH_INFO: `/v3/projects/28e45518605b42fe913814568d2a07aa/users/52841df016f0464bbcdb52790bdda306/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:50.769350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cad78-a16e-4e36-8091-4e99a97b934c admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e45518605b42fe913814568d2a07aa, user_id=52841df016f0464bbcdb52790bdda306, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:50.772272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cad78-a16e-4e36-8091-4e99a97b934c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:50.792393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1149/2301] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 20:59:50 2022] PUT /identity/v3/projects/28e45518605b42fe913814568d2a07aa/users/52841df016f0464bbcdb52790bdda306/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 20:59:50.804628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf194b-be4e-4fe0-80b6-1b9abc06a925 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:50.805283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf194b-be4e-4fe0-80b6-1b9abc06a925 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:50.805599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf194b-be4e-4fe0-80b6-1b9abc06a925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:50.846349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-81cf194b-be4e-4fe0-80b6-1b9abc06a925 None None] MFA Rules not processed for user `52841df016f0464bbcdb52790bdda306`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 20:59:50.847578 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-81cf194b-be4e-4fe0-80b6-1b9abc06a925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:50.920459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1153/2302] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 20:59:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 20:59:50.932269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cad78-a16e-4e36-8091-4e99a97b934c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 20:59:50.934125 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f71cad78-a16e-4e36-8091-4e99a97b934c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 20:59:50.996439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cad78-a16e-4e36-8091-4e99a97b934c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 20:59:51.000503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da8c1d-beeb-42d1-a068-e5576deef523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 20:59:51.001225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da8c1d-beeb-42d1-a068-e5576deef523 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 20:59:51.001567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da8c1d-beeb-42d1-a068-e5576deef523 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 20:59:51.001906 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da8c1d-beeb-42d1-a068-e5576deef523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 20:59:51.011574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da8c1d-beeb-42d1-a068-e5576deef523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 20:59:51.013319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da8c1d-beeb-42d1-a068-e5576deef523 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 20:59:51.023253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1150/2303] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 20:59:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:00:00.917568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:01.019405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:09.657890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c10b3-116a-42e2-80e5-1320ff12babd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:09.658336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c10b3-116a-42e2-80e5-1320ff12babd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:09.658435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c10b3-116a-42e2-80e5-1320ff12babd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:09.677188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-664c10b3-116a-42e2-80e5-1320ff12babd None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:09.678377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-664c10b3-116a-42e2-80e5-1320ff12babd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:09.737435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1154/2304] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:00:09.792320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da8c1d-beeb-42d1-a068-e5576deef523 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:09.793998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-05da8c1d-beeb-42d1-a068-e5576deef523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:09.875248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da8c1d-beeb-42d1-a068-e5576deef523 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:09.880493 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57185fa5-6070-45ab-bddb-38498cc44147 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:09.881159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57185fa5-6070-45ab-bddb-38498cc44147 service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:09.881445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57185fa5-6070-45ab-bddb-38498cc44147 service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:09.881722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57185fa5-6070-45ab-bddb-38498cc44147 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:09.891525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57185fa5-6070-45ab-bddb-38498cc44147 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:09.893172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57185fa5-6070-45ab-bddb-38498cc44147 service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:09.902007 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1151/2305] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:00:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:00:19.729164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=25 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:19.898869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=32 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:24.820684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c10b3-116a-42e2-80e5-1320ff12babd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:24.901839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c10b3-116a-42e2-80e5-1320ff12babd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:24.906827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:24.907842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:24.908302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:24.908520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] PATH_INFO: `/v3/users/52841df016f0464bbcdb52790bdda306` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:24.909691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52841df016f0464bbcdb52790bdda306)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:24.911698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:24.975838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52841df016f0464bbcdb52790bdda306'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:24.986955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52841df016f0464bbcdb52790bdda306'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:24.988226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52841df016f0464bbcdb52790bdda306'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:25.000996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52841df016f0464bbcdb52790bdda306'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:25.015784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52841df016f0464bbcdb52790bdda306'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:25.026766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1155/2306] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:00:24 2022] DELETE /identity/v3/users/52841df016f0464bbcdb52790bdda306 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:00:25.028806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc35807-0e42-48af-977d-b715030eece1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:25.029732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc35807-0e42-48af-977d-b715030eece1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:25.030396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc35807-0e42-48af-977d-b715030eece1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:25.053375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5fc35807-0e42-48af-977d-b715030eece1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:25.055393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5fc35807-0e42-48af-977d-b715030eece1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:25.138831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1152/2307] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:00:25.150012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:25.151733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:25.229774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb003b-bc91-48f8-9b2c-6cb28df2a9b7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:25.233967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f1321-0d38-4c71-8744-86d1162fe4e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:25.234786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f1321-0d38-4c71-8744-86d1162fe4e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:25.235349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f1321-0d38-4c71-8744-86d1162fe4e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:25.235772 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f1321-0d38-4c71-8744-86d1162fe4e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:25.246749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f1321-0d38-4c71-8744-86d1162fe4e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:25.248463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f1321-0d38-4c71-8744-86d1162fe4e8 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:25.263481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1156/2308] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:00:25.649595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc35807-0e42-48af-977d-b715030eece1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:25.651302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5fc35807-0e42-48af-977d-b715030eece1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:25.672616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc35807-0e42-48af-977d-b715030eece1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:25.686478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:25.687891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:25.688439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:25.688916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] PATH_INFO: `/v3/projects/28e45518605b42fe913814568d2a07aa` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:25.691958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=28e45518605b42fe913814568d2a07aa)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:25.695155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:25.793698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28e45518605b42fe913814568d2a07aa', 'request_id': 'req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:25.801942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28e45518605b42fe913814568d2a07aa', 'request_id': 'req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:25.820990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28e45518605b42fe913814568d2a07aa', 'request_id': 'req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:25.838294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] The token cache is being invalidate because project 28e45518605b42fe913814568d2a07aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:00:25.838615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:25.839306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b9a0f86f-c113-46fc-8aee-fc3e70f1de8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:25.845878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1153/2309] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:00:25 2022] DELETE /identity/v3/projects/28e45518605b42fe913814568d2a07aa => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:00:25.847780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70097a-3154-4bb6-8a3a-6ae37fab7ab5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:25.848653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70097a-3154-4bb6-8a3a-6ae37fab7ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:25.849171 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70097a-3154-4bb6-8a3a-6ae37fab7ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:25.887108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-db70097a-3154-4bb6-8a3a-6ae37fab7ab5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:25.888519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-db70097a-3154-4bb6-8a3a-6ae37fab7ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:25.924741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:26.031458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1157/2310] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:00:26.039536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcd2b8-c3ce-4b79-9873-b8db41163c11 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:26.041285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcd2b8-c3ce-4b79-9873-b8db41163c11 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:26.041770 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcd2b8-c3ce-4b79-9873-b8db41163c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:26.071677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8ddcd2b8-c3ce-4b79-9873-b8db41163c11 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:26.073670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8ddcd2b8-c3ce-4b79-9873-b8db41163c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:26.088940 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1154/2311] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:00:26.089511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70097a-3154-4bb6-8a3a-6ae37fab7ab5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:26.103675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70097a-3154-4bb6-8a3a-6ae37fab7ab5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:26.110437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9594721-37c2-41d9-bb59-f722f35282ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:26.111593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9594721-37c2-41d9-bb59-f722f35282ae admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:26.111957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9594721-37c2-41d9-bb59-f722f35282ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:26.112526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9594721-37c2-41d9-bb59-f722f35282ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:26.113635 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9594721-37c2-41d9-bb59-f722f35282ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:00:26.114102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9594721-37c2-41d9-bb59-f722f35282ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:26.117440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9594721-37c2-41d9-bb59-f722f35282ae admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:26.143784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcd2b8-c3ce-4b79-9873-b8db41163c11 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:26.144335 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1158/2312] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:00:26.158500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcd2b8-c3ce-4b79-9873-b8db41163c11 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:26.166041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794f2b4-b331-4701-b005-7d0dafa7f237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:26.167088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794f2b4-b331-4701-b005-7d0dafa7f237 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:26.167475 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794f2b4-b331-4701-b005-7d0dafa7f237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:26.167868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794f2b4-b331-4701-b005-7d0dafa7f237 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:26.168844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794f2b4-b331-4701-b005-7d0dafa7f237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:26.171094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794f2b4-b331-4701-b005-7d0dafa7f237 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:26.212767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1155/2313] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:00:26 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:00:26.219964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9594721-37c2-41d9-bb59-f722f35282ae admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:26.234900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9594721-37c2-41d9-bb59-f722f35282ae admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:26.342461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc908d-9f21-4681-981a-9a0fc01295fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:26.343172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc908d-9f21-4681-981a-9a0fc01295fc admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:26.343457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc908d-9f21-4681-981a-9a0fc01295fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:26.343731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc908d-9f21-4681-981a-9a0fc01295fc admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:26.344479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc908d-9f21-4681-981a-9a0fc01295fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:26.354882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc908d-9f21-4681-981a-9a0fc01295fc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:26.388540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1159/2314] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:00:26 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:00:26.392242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794f2b4-b331-4701-b005-7d0dafa7f237 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:26.422143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794f2b4-b331-4701-b005-7d0dafa7f237 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:26.443978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb069be9-d64a-4045-ae1e-f027f856b442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:26.443978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb069be9-d64a-4045-ae1e-f027f856b442 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:26.443978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb069be9-d64a-4045-ae1e-f027f856b442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:26.443978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb069be9-d64a-4045-ae1e-f027f856b442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:26.446077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb069be9-d64a-4045-ae1e-f027f856b442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:26.446077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb069be9-d64a-4045-ae1e-f027f856b442 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:26.456249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1156/2315] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:00:26.457845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc908d-9f21-4681-981a-9a0fc01295fc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:26.488273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc908d-9f21-4681-981a-9a0fc01295fc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:26.493732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1c9cb-079d-43e4-a55b-43da7a220bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:26.495087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1c9cb-079d-43e4-a55b-43da7a220bda admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:26.495531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1c9cb-079d-43e4-a55b-43da7a220bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:26.495948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1c9cb-079d-43e4-a55b-43da7a220bda admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:26.496916 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1c9cb-079d-43e4-a55b-43da7a220bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:26.499317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1c9cb-079d-43e4-a55b-43da7a220bda admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:26.522442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1160/2316] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:00:26.523796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb069be9-d64a-4045-ae1e-f027f856b442 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:26.589258 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb069be9-d64a-4045-ae1e-f027f856b442 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:26.594564 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e314a-ec02-4fbd-800a-f256bfa146c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:26.595292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e314a-ec02-4fbd-800a-f256bfa146c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:26.595575 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e314a-ec02-4fbd-800a-f256bfa146c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:26.596138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e314a-ec02-4fbd-800a-f256bfa146c3 admin admin] PATH_INFO: `/v3/projects/f4f4bbce7b684e7fbf3ba1491cd88ab1/users/5c0aef182e254474b549b6bb11da61c6/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:26.611872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e314a-ec02-4fbd-800a-f256bfa146c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f4bbce7b684e7fbf3ba1491cd88ab1, user_id=5c0aef182e254474b549b6bb11da61c6, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:26.615049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e314a-ec02-4fbd-800a-f256bfa146c3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:26.629741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1157/2317] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:00:26 2022] PUT /identity/v3/projects/f4f4bbce7b684e7fbf3ba1491cd88ab1/users/5c0aef182e254474b549b6bb11da61c6/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:00:26.642390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e2856-9bdd-42da-8596-8ea85e9c6998 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:26.644800 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e2856-9bdd-42da-8596-8ea85e9c6998 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:26.644902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e2856-9bdd-42da-8596-8ea85e9c6998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:26.703646 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-445e2856-9bdd-42da-8596-8ea85e9c6998 None None] MFA Rules not processed for user `5c0aef182e254474b549b6bb11da61c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:26.704943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-445e2856-9bdd-42da-8596-8ea85e9c6998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:26.865051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1161/2318] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:00:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:00:26.880905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e314a-ec02-4fbd-800a-f256bfa146c3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:26.883070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-818e314a-ec02-4fbd-800a-f256bfa146c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:26.934837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:27.026702 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e314a-ec02-4fbd-800a-f256bfa146c3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:27.032643 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead7930-ee36-4574-b907-fb9c6305f6dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:27.033462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead7930-ee36-4574-b907-fb9c6305f6dd service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:27.034966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead7930-ee36-4574-b907-fb9c6305f6dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:27.035275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead7930-ee36-4574-b907-fb9c6305f6dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:27.047179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead7930-ee36-4574-b907-fb9c6305f6dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:27.051361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead7930-ee36-4574-b907-fb9c6305f6dd service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:27.064607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1158/2319] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:00:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:00:27.109915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e2856-9bdd-42da-8596-8ea85e9c6998 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:27.156231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:27.226128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e2856-9bdd-42da-8596-8ea85e9c6998 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:27.232322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:27.233673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:27.234001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:27.234272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] PATH_INFO: `/v3/users/5c0aef182e254474b549b6bb11da61c6` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:27.236802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c0aef182e254474b549b6bb11da61c6)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:27.239330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:27.304874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c0aef182e254474b549b6bb11da61c6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:27.314760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c0aef182e254474b549b6bb11da61c6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:27.315634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c0aef182e254474b549b6bb11da61c6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:27.332216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c0aef182e254474b549b6bb11da61c6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:27.354322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c0aef182e254474b549b6bb11da61c6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:27.368982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1162/2320] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:00:27 2022] DELETE /identity/v3/users/5c0aef182e254474b549b6bb11da61c6 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:00:27.370947 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f1d0-7ad4-47fe-8d51-2b9386ddf46c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:27.371239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f1d0-7ad4-47fe-8d51-2b9386ddf46c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:27.371555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f1d0-7ad4-47fe-8d51-2b9386ddf46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:27.398302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-00d8f1d0-7ad4-47fe-8d51-2b9386ddf46c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:27.399459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-00d8f1d0-7ad4-47fe-8d51-2b9386ddf46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:27.536275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1159/2321] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:00:27.549053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:27.549053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:27.653471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6061450-b009-4ad6-af9e-ec1d615e6b8b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:27.659823 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609096c0-339e-4038-bbcc-a0c1410d04bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:27.660737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609096c0-339e-4038-bbcc-a0c1410d04bf service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:27.661025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609096c0-339e-4038-bbcc-a0c1410d04bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:27.661309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609096c0-339e-4038-bbcc-a0c1410d04bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:27.673452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609096c0-339e-4038-bbcc-a0c1410d04bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:27.675752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609096c0-339e-4038-bbcc-a0c1410d04bf service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:27.717813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1163/2322] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:00:28.305850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f1d0-7ad4-47fe-8d51-2b9386ddf46c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:28.308101 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-00d8f1d0-7ad4-47fe-8d51-2b9386ddf46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:28.341867 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f1d0-7ad4-47fe-8d51-2b9386ddf46c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:28.347776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:28.348726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:28.348971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:28.349385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] PATH_INFO: `/v3/projects/f4f4bbce7b684e7fbf3ba1491cd88ab1` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:28.350960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f4bbce7b684e7fbf3ba1491cd88ab1)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:28.363735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:28.450149 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f4bbce7b684e7fbf3ba1491cd88ab1', 'request_id': 'req-cfd8590d-801e-465a-8436-57c5a97c9aa9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:28.459941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f4bbce7b684e7fbf3ba1491cd88ab1', 'request_id': 'req-cfd8590d-801e-465a-8436-57c5a97c9aa9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:28.477526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f4bbce7b684e7fbf3ba1491cd88ab1', 'request_id': 'req-cfd8590d-801e-465a-8436-57c5a97c9aa9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:28.506788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] The token cache is being invalidate because project f4f4bbce7b684e7fbf3ba1491cd88ab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:00:28.507118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:28.508573 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cfd8590d-801e-465a-8436-57c5a97c9aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:28.510318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1160/2323] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:00:28 2022] DELETE /identity/v3/projects/f4f4bbce7b684e7fbf3ba1491cd88ab1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:00:28.515503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe80f8-b4e8-4dbd-84a2-9493d5a001d1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:28.516132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe80f8-b4e8-4dbd-84a2-9493d5a001d1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:28.516132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe80f8-b4e8-4dbd-84a2-9493d5a001d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:28.538411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f1fe80f8-b4e8-4dbd-84a2-9493d5a001d1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:28.539681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f1fe80f8-b4e8-4dbd-84a2-9493d5a001d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:28.594959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:28.656156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1164/2324] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:00:28.659267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753f2d4-6513-4a30-9d50-1ded1b2ebd3a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:28.659561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753f2d4-6513-4a30-9d50-1ded1b2ebd3a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:28.659839 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753f2d4-6513-4a30-9d50-1ded1b2ebd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:28.682750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0753f2d4-6513-4a30-9d50-1ded1b2ebd3a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:28.684186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0753f2d4-6513-4a30-9d50-1ded1b2ebd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:28.696300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1161/2325] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:00:28.700337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe80f8-b4e8-4dbd-84a2-9493d5a001d1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:28.711488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe80f8-b4e8-4dbd-84a2-9493d5a001d1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:28.716211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a27b4-4209-4d32-b544-ce46aa9bf644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:28.717094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a27b4-4209-4d32-b544-ce46aa9bf644 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:28.717212 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a27b4-4209-4d32-b544-ce46aa9bf644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:28.717537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a27b4-4209-4d32-b544-ce46aa9bf644 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:28.718700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a27b4-4209-4d32-b544-ce46aa9bf644 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:00:28.718953 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a27b4-4209-4d32-b544-ce46aa9bf644 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:28.720625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a27b4-4209-4d32-b544-ce46aa9bf644 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:28.733524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1165/2326] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 21:00:28.737877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753f2d4-6513-4a30-9d50-1ded1b2ebd3a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:28.747781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753f2d4-6513-4a30-9d50-1ded1b2ebd3a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:28.755365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8db28-b6ad-415b-ad6b-f3b1d12aeefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:28.756078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8db28-b6ad-415b-ad6b-f3b1d12aeefd admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:28.756497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8db28-b6ad-415b-ad6b-f3b1d12aeefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:28.756858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8db28-b6ad-415b-ad6b-f3b1d12aeefd admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:28.757684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8db28-b6ad-415b-ad6b-f3b1d12aeefd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:28.760721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8db28-b6ad-415b-ad6b-f3b1d12aeefd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:28.794938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1162/2327] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:00:28 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:00:28.796391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a27b4-4209-4d32-b544-ce46aa9bf644 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:28.808552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a27b4-4209-4d32-b544-ce46aa9bf644 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:28.904463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9c336-36a7-4c0c-9a9d-18a3143294f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:28.905254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9c336-36a7-4c0c-9a9d-18a3143294f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:28.905602 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9c336-36a7-4c0c-9a9d-18a3143294f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:28.905946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9c336-36a7-4c0c-9a9d-18a3143294f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:28.906760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9c336-36a7-4c0c-9a9d-18a3143294f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:28.908935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9c336-36a7-4c0c-9a9d-18a3143294f1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:28.934164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1166/2328] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:00:28 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:00:28.938190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8db28-b6ad-415b-ad6b-f3b1d12aeefd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:28.949345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8db28-b6ad-415b-ad6b-f3b1d12aeefd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:28.954181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138f349-0bb0-4d31-a7fe-000a0e353480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:28.954790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138f349-0bb0-4d31-a7fe-000a0e353480 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:28.955097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138f349-0bb0-4d31-a7fe-000a0e353480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:28.955374 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138f349-0bb0-4d31-a7fe-000a0e353480 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:28.956135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138f349-0bb0-4d31-a7fe-000a0e353480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:28.957750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138f349-0bb0-4d31-a7fe-000a0e353480 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:28.970985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1163/2329] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:00:28.976042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9c336-36a7-4c0c-9a9d-18a3143294f1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:28.986433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9c336-36a7-4c0c-9a9d-18a3143294f1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:28.991954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4baf1d8-d45e-4fbc-a228-a415492d9820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:28.992737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baf1d8-d45e-4fbc-a228-a415492d9820 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:28.993176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baf1d8-d45e-4fbc-a228-a415492d9820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:28.993438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baf1d8-d45e-4fbc-a228-a415492d9820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:28.994197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4baf1d8-d45e-4fbc-a228-a415492d9820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:28.995810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4baf1d8-d45e-4fbc-a228-a415492d9820 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:29.017385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1167/2330] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:00:29.026676 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138f349-0bb0-4d31-a7fe-000a0e353480 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:29.044925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138f349-0bb0-4d31-a7fe-000a0e353480 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:29.050646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94064976-02c8-48a2-afe5-8ab8bd0fd7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:29.051564 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94064976-02c8-48a2-afe5-8ab8bd0fd7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:29.051823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94064976-02c8-48a2-afe5-8ab8bd0fd7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:29.052094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94064976-02c8-48a2-afe5-8ab8bd0fd7c2 admin admin] PATH_INFO: `/v3/projects/6ed25b82be2043b188d2682a5c01490a/users/5f8c2e5e6c1b4833b0e9af0de8814d13/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:29.055095 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:29.070499 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94064976-02c8-48a2-afe5-8ab8bd0fd7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed25b82be2043b188d2682a5c01490a, user_id=5f8c2e5e6c1b4833b0e9af0de8814d13, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:29.073233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94064976-02c8-48a2-afe5-8ab8bd0fd7c2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:29.087030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1164/2331] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:00:29 2022] PUT /identity/v3/projects/6ed25b82be2043b188d2682a5c01490a/users/5f8c2e5e6c1b4833b0e9af0de8814d13/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:00:29.092181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29d4f3-7591-41e6-ac67-944823cf0cd9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:29.092495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29d4f3-7591-41e6-ac67-944823cf0cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:29.092835 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29d4f3-7591-41e6-ac67-944823cf0cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:29.111693 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bb29d4f3-7591-41e6-ac67-944823cf0cd9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:29.112941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bb29d4f3-7591-41e6-ac67-944823cf0cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:29.179778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1168/2332] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:00:29.189687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94064976-02c8-48a2-afe5-8ab8bd0fd7c2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:29.192377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-94064976-02c8-48a2-afe5-8ab8bd0fd7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:29.258132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94064976-02c8-48a2-afe5-8ab8bd0fd7c2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:29.263247 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accaa4be-ef37-4fa1-9b76-edb34b7b8e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:29.264081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accaa4be-ef37-4fa1-9b76-edb34b7b8e07 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:29.264403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accaa4be-ef37-4fa1-9b76-edb34b7b8e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:29.264690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accaa4be-ef37-4fa1-9b76-edb34b7b8e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:29.277080 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accaa4be-ef37-4fa1-9b76-edb34b7b8e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:29.279335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accaa4be-ef37-4fa1-9b76-edb34b7b8e07 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:29.290618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1165/2333] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:00:32.940925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291c281-4169-49f4-a9a2-6b276c0fb7d2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:32.941818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291c281-4169-49f4-a9a2-6b276c0fb7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:32.942882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291c281-4169-49f4-a9a2-6b276c0fb7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:32.966363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6291c281-4169-49f4-a9a2-6b276c0fb7d2 None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:32.968177 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6291c281-4169-49f4-a9a2-6b276c0fb7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:33.030794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:33.048958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1169/2334] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jul 11 21:00:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:00:33.075686 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accaa4be-ef37-4fa1-9b76-edb34b7b8e07 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:33.077844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-accaa4be-ef37-4fa1-9b76-edb34b7b8e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:33.187649 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accaa4be-ef37-4fa1-9b76-edb34b7b8e07 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:33.193556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62247ef4-13f7-4b77-abef-8e049062f8c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:33.194431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62247ef4-13f7-4b77-abef-8e049062f8c3 service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:33.194798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62247ef4-13f7-4b77-abef-8e049062f8c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:33.195173 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62247ef4-13f7-4b77-abef-8e049062f8c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:33.205505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62247ef4-13f7-4b77-abef-8e049062f8c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:33.208451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62247ef4-13f7-4b77-abef-8e049062f8c3 service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:33.216897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1166/2335] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:00:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:00:33.755140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742bb6-790d-4a94-981a-583c41cf1544 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:33.755825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742bb6-790d-4a94-981a-583c41cf1544 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:33.756132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742bb6-790d-4a94-981a-583c41cf1544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:33.806459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b9742bb6-790d-4a94-981a-583c41cf1544 None None] MFA Rules not processed for user `5f8c2e5e6c1b4833b0e9af0de8814d13`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:33.807719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b9742bb6-790d-4a94-981a-583c41cf1544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:33.894289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1170/2336] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:00:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:00:33.898815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62247ef4-13f7-4b77-abef-8e049062f8c3 service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:33.909934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62247ef4-13f7-4b77-abef-8e049062f8c3 service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:33.915513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d937d-f773-4e20-98ab-cb565c1941bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:33.916213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d937d-f773-4e20-98ab-cb565c1941bc admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:33.916672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d937d-f773-4e20-98ab-cb565c1941bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:33.916946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d937d-f773-4e20-98ab-cb565c1941bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:33.917661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d937d-f773-4e20-98ab-cb565c1941bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:33.919580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d937d-f773-4e20-98ab-cb565c1941bc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:33.949789 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1167/2337] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:00:33 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:00:33.953738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742bb6-790d-4a94-981a-583c41cf1544 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:33.967307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742bb6-790d-4a94-981a-583c41cf1544 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:34.054990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cd9f8-62b1-4b58-aa42-e17cf8634321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:34.055917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cd9f8-62b1-4b58-aa42-e17cf8634321 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:34.056343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cd9f8-62b1-4b58-aa42-e17cf8634321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:34.056747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cd9f8-62b1-4b58-aa42-e17cf8634321 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:34.057528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cd9f8-62b1-4b58-aa42-e17cf8634321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:34.059564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cd9f8-62b1-4b58-aa42-e17cf8634321 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:34.081471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1171/2338] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:00:33 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:00:34.085743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d937d-f773-4e20-98ab-cb565c1941bc admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:34.096433 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d937d-f773-4e20-98ab-cb565c1941bc admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:34.101870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b67bd7-89ca-4c69-bb73-4a7562e365f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:34.102751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b67bd7-89ca-4c69-bb73-4a7562e365f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:34.103155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b67bd7-89ca-4c69-bb73-4a7562e365f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:34.103494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b67bd7-89ca-4c69-bb73-4a7562e365f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:34.104507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b67bd7-89ca-4c69-bb73-4a7562e365f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:34.106284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b67bd7-89ca-4c69-bb73-4a7562e365f9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:34.119097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1168/2339] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:00:34.123506 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cd9f8-62b1-4b58-aa42-e17cf8634321 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:34.134647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cd9f8-62b1-4b58-aa42-e17cf8634321 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:34.139921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac00f5-54d0-472b-baa2-84bb5bf5e716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:34.141850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac00f5-54d0-472b-baa2-84bb5bf5e716 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:34.142279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac00f5-54d0-472b-baa2-84bb5bf5e716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:34.142695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac00f5-54d0-472b-baa2-84bb5bf5e716 admin admin] PATH_INFO: `/v3/projects/ba0054f6e68e46749271aa7af502c931/users/8bd59999ed69444aaeb3fe60c4befcca/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:34.159317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac00f5-54d0-472b-baa2-84bb5bf5e716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0054f6e68e46749271aa7af502c931, user_id=8bd59999ed69444aaeb3fe60c4befcca, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:34.162328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac00f5-54d0-472b-baa2-84bb5bf5e716 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:34.177607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1172/2340] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:00:34 2022] PUT /identity/v3/projects/ba0054f6e68e46749271aa7af502c931/users/8bd59999ed69444aaeb3fe60c4befcca/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:00:34.182081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b67bd7-89ca-4c69-bb73-4a7562e365f9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:34.193916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b67bd7-89ca-4c69-bb73-4a7562e365f9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:34.239248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:34.257931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9196e-f7df-486a-86dd-824d63b6f421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:34.258613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9196e-f7df-486a-86dd-824d63b6f421 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:34.258896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9196e-f7df-486a-86dd-824d63b6f421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:34.259165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9196e-f7df-486a-86dd-824d63b6f421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:34.259905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9196e-f7df-486a-86dd-824d63b6f421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:34.262186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9196e-f7df-486a-86dd-824d63b6f421 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:34.273842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1169/2341] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:00:34.277931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac00f5-54d0-472b-baa2-84bb5bf5e716 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:34.288726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac00f5-54d0-472b-baa2-84bb5bf5e716 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:34.293366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da419d20-a959-4a95-a405-dc74fbae3a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:34.294263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419d20-a959-4a95-a405-dc74fbae3a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:34.294540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419d20-a959-4a95-a405-dc74fbae3a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:34.294812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419d20-a959-4a95-a405-dc74fbae3a39 admin admin] PATH_INFO: `/v3/projects/ba0054f6e68e46749271aa7af502c931/users/8bd59999ed69444aaeb3fe60c4befcca/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:34.296842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da419d20-a959-4a95-a405-dc74fbae3a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0054f6e68e46749271aa7af502c931, user_id=8bd59999ed69444aaeb3fe60c4befcca, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:34.299523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da419d20-a959-4a95-a405-dc74fbae3a39 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:34.311251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1173/2342] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:00:34 2022] PUT /identity/v3/projects/ba0054f6e68e46749271aa7af502c931/users/8bd59999ed69444aaeb3fe60c4befcca/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:00:36.012317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9196e-f7df-486a-86dd-824d63b6f421 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:36.014628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5db9196e-f7df-486a-86dd-824d63b6f421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:36.032993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9196e-f7df-486a-86dd-824d63b6f421 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:36.126435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:36.127354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:36.127688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:36.127908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] PATH_INFO: `/v3/users/934b314c46824d5397a0ae92bb07b725` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:36.129203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] RBAC: Authorizing `identity:delete_user(user_id=934b314c46824d5397a0ae92bb07b725)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:36.131579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:36.196193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934b314c46824d5397a0ae92bb07b725'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:36.196910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934b314c46824d5397a0ae92bb07b725'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:36.206233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934b314c46824d5397a0ae92bb07b725'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:36.206987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934b314c46824d5397a0ae92bb07b725'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:36.228788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8f1a276b-c1d7-47c4-8cbd-71f1516f055f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934b314c46824d5397a0ae92bb07b725'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:36.249208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1170/2343] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:00:36 2022] DELETE /identity/v3/users/934b314c46824d5397a0ae92bb07b725 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:00:36.519834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da419d20-a959-4a95-a405-dc74fbae3a39 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:36.523343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-da419d20-a959-4a95-a405-dc74fbae3a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:36.552453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da419d20-a959-4a95-a405-dc74fbae3a39 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:36.615263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:36.674563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:36.676626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:36.677324 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:36.678695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] PATH_INFO: `/v3/projects/804c8a0ad1674b818f8bbc53d661284b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:36.680920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=804c8a0ad1674b818f8bbc53d661284b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:36.683257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:36.743008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804c8a0ad1674b818f8bbc53d661284b', 'request_id': 'req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:36.750525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804c8a0ad1674b818f8bbc53d661284b', 'request_id': 'req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:36.760676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804c8a0ad1674b818f8bbc53d661284b', 'request_id': 'req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:36.778102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] The token cache is being invalidate because project 804c8a0ad1674b818f8bbc53d661284b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:00:36.778522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:36.779609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:00:36.781734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1174/2344] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:00:36 2022] DELETE /identity/v3/projects/804c8a0ad1674b818f8bbc53d661284b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:00:39.139364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc2e1-b0fd-4ece-968f-e50816f24795 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:39.139808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc2e1-b0fd-4ece-968f-e50816f24795 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:39.139907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc2e1-b0fd-4ece-968f-e50816f24795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:39.167694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-057cc2e1-b0fd-4ece-968f-e50816f24795 None None] MFA Rules not processed for user `8bd59999ed69444aaeb3fe60c4befcca`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:00:39.168848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-057cc2e1-b0fd-4ece-968f-e50816f24795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:39.219989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:39.248243 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1171/2345] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:00:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:00:39.265549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:00:39.267185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:39.341515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5d5d8-efb3-4483-ab1a-abfefed8dc1e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:00:39.346933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce3d27-33e2-4b50-a1d9-fec2f22f367f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:00:39.347596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3d27-33e2-4b50-a1d9-fec2f22f367f service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:00:39.347881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3d27-33e2-4b50-a1d9-fec2f22f367f service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:00:39.348190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3d27-33e2-4b50-a1d9-fec2f22f367f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:00:39.350168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-40ce3d27-33e2-4b50-a1d9-fec2f22f367f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:00:39.414667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce3d27-33e2-4b50-a1d9-fec2f22f367f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:00:39.416216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce3d27-33e2-4b50-a1d9-fec2f22f367f service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:00:39.421278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:00:39.429549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1175/2346] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:00:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 21:00:49.244538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=8 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:01:19.124531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e2c6-80af-4008-8681-fce4e1bceb0e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:01:19.125189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e2c6-80af-4008-8681-fce4e1bceb0e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:01:19.125189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e2c6-80af-4008-8681-fce4e1bceb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:01:19.143645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b015e2c6-80af-4008-8681-fce4e1bceb0e None None] MFA Rules not processed for user `5f8c2e5e6c1b4833b0e9af0de8814d13`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:01:19.144862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b015e2c6-80af-4008-8681-fce4e1bceb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:01:19.152010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1172/2347] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:01:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:01:20.549189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce3d27-33e2-4b50-a1d9-fec2f22f367f service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:01:20.550792 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-40ce3d27-33e2-4b50-a1d9-fec2f22f367f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:01:20.624383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce3d27-33e2-4b50-a1d9-fec2f22f367f service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:01:20.629733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd57c3f-f505-43c3-90f8-bf190c890dc2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:01:20.630383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57c3f-f505-43c3-90f8-bf190c890dc2 service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:01:20.630663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57c3f-f505-43c3-90f8-bf190c890dc2 service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:01:20.630936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57c3f-f505-43c3-90f8-bf190c890dc2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:01:20.632817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-acd57c3f-f505-43c3-90f8-bf190c890dc2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:01:20.644707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd57c3f-f505-43c3-90f8-bf190c890dc2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:01:20.646305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd57c3f-f505-43c3-90f8-bf190c890dc2 service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:01:20.656450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1176/2348] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:01:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:01:29.138078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:01:30.652286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=32 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:01:35.631845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712e6a9-2f04-4bc9-91a0-524f2b7972a1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:01:35.632417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712e6a9-2f04-4bc9-91a0-524f2b7972a1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:01:35.632417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712e6a9-2f04-4bc9-91a0-524f2b7972a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:01:35.651993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a712e6a9-2f04-4bc9-91a0-524f2b7972a1 None None] MFA Rules not processed for user `63ec8ab7842a443c922a39f5ffcfd51a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:01:35.653222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a712e6a9-2f04-4bc9-91a0-524f2b7972a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:01:35.724799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1173/2349] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:01:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:01:45.714579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=25 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:01:49.863761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd57c3f-f505-43c3-90f8-bf190c890dc2 service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:01:49.865385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-acd57c3f-f505-43c3-90f8-bf190c890dc2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:01:49.927224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd57c3f-f505-43c3-90f8-bf190c890dc2 service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:01:49.932070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4752f-2ddc-432d-b9f4-bed6128b332a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:01:49.932750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4752f-2ddc-432d-b9f4-bed6128b332a service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:01:49.933033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4752f-2ddc-432d-b9f4-bed6128b332a service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:01:49.933304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4752f-2ddc-432d-b9f4-bed6128b332a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:01:49.943555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4752f-2ddc-432d-b9f4-bed6128b332a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:01:49.945065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4752f-2ddc-432d-b9f4-bed6128b332a service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:01:49.955148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1177/2350] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:01:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 21:01:56.723902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27083a13-7203-45ef-8c98-0dbad46cc06f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:01:56.724454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27083a13-7203-45ef-8c98-0dbad46cc06f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:01:56.724454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27083a13-7203-45ef-8c98-0dbad46cc06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:01:56.744804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-27083a13-7203-45ef-8c98-0dbad46cc06f None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:01:56.746145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-27083a13-7203-45ef-8c98-0dbad46cc06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:01:56.824574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1174/2351] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 21:01:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:01:56.875895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4752f-2ddc-432d-b9f4-bed6128b332a service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:01:56.877740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a2b4752f-2ddc-432d-b9f4-bed6128b332a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:01:56.951985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4752f-2ddc-432d-b9f4-bed6128b332a service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:01:56.957707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e670df-d66a-4458-b524-a5be5afce083 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:01:56.958367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e670df-d66a-4458-b524-a5be5afce083 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:01:56.958674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e670df-d66a-4458-b524-a5be5afce083 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:01:56.958946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e670df-d66a-4458-b524-a5be5afce083 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:01:56.967647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e670df-d66a-4458-b524-a5be5afce083 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:01:56.969458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e670df-d66a-4458-b524-a5be5afce083 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:01:56.978157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1178/2352] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:01:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:02:06.817973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=25 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:02:06.975005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=58 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:02:17.678208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e7e9a-3b6b-4ca7-9faf-9871f046e019 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:17.678762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e7e9a-3b6b-4ca7-9faf-9871f046e019 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:17.678762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e7e9a-3b6b-4ca7-9faf-9871f046e019 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:17.680183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1175/2353] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 21:02:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 21:02:17.688722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f079ada-7ecd-43f1-a84c-a68dcfdf2b98 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:17.689900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f079ada-7ecd-43f1-a84c-a68dcfdf2b98 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:17.690456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f079ada-7ecd-43f1-a84c-a68dcfdf2b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:17.710231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6f079ada-7ecd-43f1-a84c-a68dcfdf2b98 None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:02:17.711460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6f079ada-7ecd-43f1-a84c-a68dcfdf2b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:02:17.720826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1179/2354] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 21:02:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:02:17.732792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e7e9a-3b6b-4ca7-9faf-9871f046e019 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:02:17.734569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-884e7e9a-3b6b-4ca7-9faf-9871f046e019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:02:17.792828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e7e9a-3b6b-4ca7-9faf-9871f046e019 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:02:17.796586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a791599-399c-4196-a188-0b5155cc8fa2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097b12fbf6494336b11ece5c498a4103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:02:17.797244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a791599-399c-4196-a188-0b5155cc8fa2 service glance] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:17.797526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a791599-399c-4196-a188-0b5155cc8fa2 service glance] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:17.797801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a791599-399c-4196-a188-0b5155cc8fa2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:17.809033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a791599-399c-4196-a188-0b5155cc8fa2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:02:17.813373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a791599-399c-4196-a188-0b5155cc8fa2 service glance] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:02:17.824999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1176/2355] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:02:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:02:17.892832 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cde034-c5ef-449c-91b4-4b7c6391c46e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:17.893043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cde034-c5ef-449c-91b4-4b7c6391c46e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:17.893320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cde034-c5ef-449c-91b4-4b7c6391c46e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:17.894708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1180/2356] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 21:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 21:02:17.899711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b0b4f-0e43-44bb-8e80-1cef41f41d83 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:17.900004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b0b4f-0e43-44bb-8e80-1cef41f41d83 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:17.900318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b0b4f-0e43-44bb-8e80-1cef41f41d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:17.919910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ac9b0b4f-0e43-44bb-8e80-1cef41f41d83 None None] MFA Rules not processed for user `0574d5c8974f43218653208ca72c43bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:02:17.921170 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ac9b0b4f-0e43-44bb-8e80-1cef41f41d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:02:17.929482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1177/2357] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 21:02:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:02:17.941337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cde034-c5ef-449c-91b4-4b7c6391c46e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:02:17.952865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cde034-c5ef-449c-91b4-4b7c6391c46e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:02:17.956753 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fe1da-339c-4f7c-8fa8-8448a5f91163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:02:17.957387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fe1da-339c-4f7c-8fa8-8448a5f91163 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:17.957665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fe1da-339c-4f7c-8fa8-8448a5f91163 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:17.957984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fe1da-339c-4f7c-8fa8-8448a5f91163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:17.965187 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fe1da-339c-4f7c-8fa8-8448a5f91163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:02:17.966626 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fe1da-339c-4f7c-8fa8-8448a5f91163 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:02:17.974555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1181/2358] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:02:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:02:18.060670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b54175-765a-44bf-bd04-4fb868bc1ac7 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:18.060965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b54175-765a-44bf-bd04-4fb868bc1ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:18.061269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b54175-765a-44bf-bd04-4fb868bc1ac7 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:18.062706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1178/2359] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 21:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 21:02:18.069092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013f1d9-875a-423d-8498-67ffeeb39d4c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:18.069411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013f1d9-875a-423d-8498-67ffeeb39d4c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:18.069689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013f1d9-875a-423d-8498-67ffeeb39d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:18.089786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1013f1d9-875a-423d-8498-67ffeeb39d4c None None] MFA Rules not processed for user `acb3ad858e4848b9a325c72c7328c863`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:02:18.090979 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1013f1d9-875a-423d-8498-67ffeeb39d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:02:18.101409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1182/2360] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 21:02:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:02:18.112956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b54175-765a-44bf-bd04-4fb868bc1ac7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:02:18.115316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-73b54175-765a-44bf-bd04-4fb868bc1ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:02:18.130376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b54175-765a-44bf-bd04-4fb868bc1ac7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:02:18.134483 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484637b7-11c4-4080-b4de-49827c808917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:02:18.135241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484637b7-11c4-4080-b4de-49827c808917 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:18.135548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484637b7-11c4-4080-b4de-49827c808917 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:18.135803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484637b7-11c4-4080-b4de-49827c808917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:18.146987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484637b7-11c4-4080-b4de-49827c808917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:02:18.148525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484637b7-11c4-4080-b4de-49827c808917 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:02:18.159498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1179/2361] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:02:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:02:20.062396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013f1d9-875a-423d-8498-67ffeeb39d4c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:02:20.074671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013f1d9-875a-423d-8498-67ffeeb39d4c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:02:20.078474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af8d63-92f2-4d58-adf2-03e5df42ebb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:02:20.079172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af8d63-92f2-4d58-adf2-03e5df42ebb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:20.079442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af8d63-92f2-4d58-adf2-03e5df42ebb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:20.079723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af8d63-92f2-4d58-adf2-03e5df42ebb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:20.081622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-74af8d63-92f2-4d58-adf2-03e5df42ebb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:02:20.095231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af8d63-92f2-4d58-adf2-03e5df42ebb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:02:20.096707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af8d63-92f2-4d58-adf2-03e5df42ebb6 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:02:20.107861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1183/2362] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:02:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:02:21.312371 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484637b7-11c4-4080-b4de-49827c808917 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:02:21.321920 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484637b7-11c4-4080-b4de-49827c808917 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:02:21.325858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37c9e3-2263-409a-8cd7-63ece3f08cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:02:21.326503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37c9e3-2263-409a-8cd7-63ece3f08cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:21.326780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37c9e3-2263-409a-8cd7-63ece3f08cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:21.327053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37c9e3-2263-409a-8cd7-63ece3f08cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:21.328983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5b37c9e3-2263-409a-8cd7-63ece3f08cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:02:21.343574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37c9e3-2263-409a-8cd7-63ece3f08cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:02:21.345068 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37c9e3-2263-409a-8cd7-63ece3f08cd7 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:02:21.357090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1180/2363] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:02:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:02:21.654160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0ee0-08af-440b-86fa-024be5af852c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:21.654643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0ee0-08af-440b-86fa-024be5af852c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:21.654851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0ee0-08af-440b-86fa-024be5af852c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:21.673472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b2fe0ee0-08af-440b-86fa-024be5af852c None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:02:21.674717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b2fe0ee0-08af-440b-86fa-024be5af852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:02:21.682856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1184/2364] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 21:02:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:02:21.695249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37c9e3-2263-409a-8cd7-63ece3f08cd7 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:02:21.706701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37c9e3-2263-409a-8cd7-63ece3f08cd7 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:02:21.712674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711bb983-103f-4ffa-9072-6e5347265a74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:02:21.713320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bb983-103f-4ffa-9072-6e5347265a74 service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:21.713600 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bb983-103f-4ffa-9072-6e5347265a74 service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:21.714846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bb983-103f-4ffa-9072-6e5347265a74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:21.723839 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711bb983-103f-4ffa-9072-6e5347265a74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:02:21.726195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711bb983-103f-4ffa-9072-6e5347265a74 service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:02:21.734841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1181/2365] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:02:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:02:31.667900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=21 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:02:31.731341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=47 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:02:36.968491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe0ee0-08af-440b-86fa-024be5af852c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:02:36.982964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe0ee0-08af-440b-86fa-024be5af852c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:02:36.989996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa7ab3-df9b-4890-976c-3267eb6424e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:02:36.990658 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa7ab3-df9b-4890-976c-3267eb6424e5 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:36.990931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa7ab3-df9b-4890-976c-3267eb6424e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:36.991199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa7ab3-df9b-4890-976c-3267eb6424e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:36.993185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c4aa7ab3-df9b-4890-976c-3267eb6424e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:02:37.063699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa7ab3-df9b-4890-976c-3267eb6424e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:02:37.065329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa7ab3-df9b-4890-976c-3267eb6424e5 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:02:37.074740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1185/2366] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:02:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:02:47.070684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=28 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:02:57.241013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711bb983-103f-4ffa-9072-6e5347265a74 service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:02:57.251244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711bb983-103f-4ffa-9072-6e5347265a74 service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:02:57.255998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e9657-7162-4597-88b3-997fde7f5594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:02:57.256771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e9657-7162-4597-88b3-997fde7f5594 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:02:57.257053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e9657-7162-4597-88b3-997fde7f5594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:02:57.257324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e9657-7162-4597-88b3-997fde7f5594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:02:57.259319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-560e9657-7162-4597-88b3-997fde7f5594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:02:57.271606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e9657-7162-4597-88b3-997fde7f5594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:02:57.273097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e9657-7162-4597-88b3-997fde7f5594 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:02:57.281676 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1182/2367] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:02:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:03:00.150578 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e26372-eb65-4fdf-93f3-563d4dd5a730 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:00.151111 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e26372-eb65-4fdf-93f3-563d4dd5a730 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:00.151111 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e26372-eb65-4fdf-93f3-563d4dd5a730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:00.169958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a9e26372-eb65-4fdf-93f3-563d4dd5a730 None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:03:00.171191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a9e26372-eb65-4fdf-93f3-563d4dd5a730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:00.180692 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1186/2368] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jul 11 21:03:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:03:00.208018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e9657-7162-4597-88b3-997fde7f5594 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:00.221997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e9657-7162-4597-88b3-997fde7f5594 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:00.228588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed01eb-c4be-473e-b68c-91fcf71055db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:00.229256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed01eb-c4be-473e-b68c-91fcf71055db service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:00.229537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed01eb-c4be-473e-b68c-91fcf71055db service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:00.229807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed01eb-c4be-473e-b68c-91fcf71055db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:00.238701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed01eb-c4be-473e-b68c-91fcf71055db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:00.240412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed01eb-c4be-473e-b68c-91fcf71055db service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:00.249354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1183/2369] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:03:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:03:09.146873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e26372-eb65-4fdf-93f3-563d4dd5a730 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:09.148598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a9e26372-eb65-4fdf-93f3-563d4dd5a730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:09.221379 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e26372-eb65-4fdf-93f3-563d4dd5a730 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:09.226742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:09.227637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:09.227925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:09.228230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] PATH_INFO: `/v3/users/63ec8ab7842a443c922a39f5ffcfd51a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:09.229568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ec8ab7842a443c922a39f5ffcfd51a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:09.231639 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:09.282874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ec8ab7842a443c922a39f5ffcfd51a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:03:09.290917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ec8ab7842a443c922a39f5ffcfd51a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:03:09.291574 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ec8ab7842a443c922a39f5ffcfd51a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:03:09.300441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ec8ab7842a443c922a39f5ffcfd51a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:03:09.313314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a94793b3-ad5a-4d5a-a4d5-7a22aa8c92e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ec8ab7842a443c922a39f5ffcfd51a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:03:09.319436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1187/2370] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:03:09 2022] DELETE /identity/v3/users/63ec8ab7842a443c922a39f5ffcfd51a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:03:09.508882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed01eb-c4be-473e-b68c-91fcf71055db service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:09.521545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed01eb-c4be-473e-b68c-91fcf71055db service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:09.579409 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:09.580319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:09.580599 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:09.580871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] PATH_INFO: `/v3/projects/36d40e24cac74460bbbed4dffce9a4da` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:09.582097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d40e24cac74460bbbed4dffce9a4da)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:09.584026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:09.631914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d40e24cac74460bbbed4dffce9a4da', 'request_id': 'req-4a6584b9-54dd-43ba-968d-7fd81d096ce6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:03:09.638330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d40e24cac74460bbbed4dffce9a4da', 'request_id': 'req-4a6584b9-54dd-43ba-968d-7fd81d096ce6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:03:09.647013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d40e24cac74460bbbed4dffce9a4da', 'request_id': 'req-4a6584b9-54dd-43ba-968d-7fd81d096ce6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:03:09.657339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] The token cache is being invalidate because project 36d40e24cac74460bbbed4dffce9a4da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:03:09.657625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:03:09.658237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4a6584b9-54dd-43ba-968d-7fd81d096ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:03:09.659826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1184/2371] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:03:09 2022] DELETE /identity/v3/projects/36d40e24cac74460bbbed4dffce9a4da => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:03:09.664896 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8c5cd-bc2f-4fc6-8233-11d2f2613fa4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:09.665189 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8c5cd-bc2f-4fc6-8233-11d2f2613fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:09.665462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8c5cd-bc2f-4fc6-8233-11d2f2613fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:09.685095 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-89c8c5cd-bc2f-4fc6-8233-11d2f2613fa4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:03:09.686400 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-89c8c5cd-bc2f-4fc6-8233-11d2f2613fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:09.749236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1188/2372] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:03:09.757187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024f8b-2371-41b2-b584-14a8d7823eba None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:09.757481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024f8b-2371-41b2-b584-14a8d7823eba None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:09.757754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024f8b-2371-41b2-b584-14a8d7823eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:09.777251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e1024f8b-2371-41b2-b584-14a8d7823eba None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:03:09.778455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e1024f8b-2371-41b2-b584-14a8d7823eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:09.787714 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1185/2373] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:03:09.792276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8c5cd-bc2f-4fc6-8233-11d2f2613fa4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:09.802855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8c5cd-bc2f-4fc6-8233-11d2f2613fa4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:09.808503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885fe68-b4c6-4476-a01e-ba207913f700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:09.809160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885fe68-b4c6-4476-a01e-ba207913f700 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:09.809438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885fe68-b4c6-4476-a01e-ba207913f700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:09.809706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885fe68-b4c6-4476-a01e-ba207913f700 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:09.810433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885fe68-b4c6-4476-a01e-ba207913f700 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:03:09.810860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885fe68-b4c6-4476-a01e-ba207913f700 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:09.812530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885fe68-b4c6-4476-a01e-ba207913f700 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:09.821502 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:03:09.828985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1189/2374] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 21:03:09.830216 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1024f8b-2371-41b2-b584-14a8d7823eba None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:09.839888 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1024f8b-2371-41b2-b584-14a8d7823eba None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:09.845046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277f5a7-be55-4b27-b8ff-18678e04dfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:09.845760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277f5a7-be55-4b27-b8ff-18678e04dfca admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:09.846041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277f5a7-be55-4b27-b8ff-18678e04dfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:09.846310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277f5a7-be55-4b27-b8ff-18678e04dfca admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:09.847003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277f5a7-be55-4b27-b8ff-18678e04dfca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:09.848953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277f5a7-be55-4b27-b8ff-18678e04dfca admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:09.876323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1186/2375] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:03:09 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:03:09.880730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885fe68-b4c6-4476-a01e-ba207913f700 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:09.891243 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885fe68-b4c6-4476-a01e-ba207913f700 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:09.947773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6fabc-33e2-4a43-99d1-5a17aa23423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:09.948470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6fabc-33e2-4a43-99d1-5a17aa23423d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:09.948746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6fabc-33e2-4a43-99d1-5a17aa23423d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:09.949022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6fabc-33e2-4a43-99d1-5a17aa23423d admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:09.949701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6fabc-33e2-4a43-99d1-5a17aa23423d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:09.951631 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6fabc-33e2-4a43-99d1-5a17aa23423d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:09.973642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1190/2376] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:03:09 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:03:09.977934 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277f5a7-be55-4b27-b8ff-18678e04dfca admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:09.989263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277f5a7-be55-4b27-b8ff-18678e04dfca admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:09.994164 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7d626-0f4b-4028-b7ad-097d7c61a029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:09.994829 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7d626-0f4b-4028-b7ad-097d7c61a029 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:09.995207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7d626-0f4b-4028-b7ad-097d7c61a029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:09.995582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7d626-0f4b-4028-b7ad-097d7c61a029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:09.996462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7d626-0f4b-4028-b7ad-097d7c61a029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:09.998075 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7d626-0f4b-4028-b7ad-097d7c61a029 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:10.008880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1187/2377] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:03:10.013044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6fabc-33e2-4a43-99d1-5a17aa23423d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:10.023386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6fabc-33e2-4a43-99d1-5a17aa23423d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:10.029560 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06863d9-eb75-4df3-b101-96a7ac1e3bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:10.030339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06863d9-eb75-4df3-b101-96a7ac1e3bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:10.030697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06863d9-eb75-4df3-b101-96a7ac1e3bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:10.030978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06863d9-eb75-4df3-b101-96a7ac1e3bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:10.031713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06863d9-eb75-4df3-b101-96a7ac1e3bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:10.033366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06863d9-eb75-4df3-b101-96a7ac1e3bcc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:10.043857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1191/2378] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:03:10.048383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7d626-0f4b-4028-b7ad-097d7c61a029 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:10.059059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7d626-0f4b-4028-b7ad-097d7c61a029 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:10.064706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b312fb4-caa7-42b6-87f4-3811bca2c42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:10.065596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b312fb4-caa7-42b6-87f4-3811bca2c42f admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:10.065868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b312fb4-caa7-42b6-87f4-3811bca2c42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:10.066154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b312fb4-caa7-42b6-87f4-3811bca2c42f admin admin] PATH_INFO: `/v3/projects/6f54283593cd4e71ba6b700377bf0b4e/users/de2fb55182604091b35826b2c2fd090c/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:10.082438 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b312fb4-caa7-42b6-87f4-3811bca2c42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f54283593cd4e71ba6b700377bf0b4e, user_id=de2fb55182604091b35826b2c2fd090c, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:10.085134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b312fb4-caa7-42b6-87f4-3811bca2c42f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:10.096064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:03:10.101692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1188/2379] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:03:10 2022] PUT /identity/v3/projects/6f54283593cd4e71ba6b700377bf0b4e/users/de2fb55182604091b35826b2c2fd090c/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:03:10.107798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c6c31-c4b1-4fe1-ac91-6ea1f9cd04b2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:10.108248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c6c31-c4b1-4fe1-ac91-6ea1f9cd04b2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:10.108548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c6c31-c4b1-4fe1-ac91-6ea1f9cd04b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:10.127417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-143c6c31-c4b1-4fe1-ac91-6ea1f9cd04b2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:03:10.128689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-143c6c31-c4b1-4fe1-ac91-6ea1f9cd04b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:10.191799 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1192/2380] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:03:10.202989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b312fb4-caa7-42b6-87f4-3811bca2c42f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:10.204871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1b312fb4-caa7-42b6-87f4-3811bca2c42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:10.274152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b312fb4-caa7-42b6-87f4-3811bca2c42f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:10.278568 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd33b7f-7a6e-4e42-a230-1cd6ff29b0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:10.279265 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd33b7f-7a6e-4e42-a230-1cd6ff29b0aa service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:10.279526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd33b7f-7a6e-4e42-a230-1cd6ff29b0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:10.279791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd33b7f-7a6e-4e42-a230-1cd6ff29b0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:10.290154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd33b7f-7a6e-4e42-a230-1cd6ff29b0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:10.291603 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd33b7f-7a6e-4e42-a230-1cd6ff29b0aa service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:10.302177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1189/2381] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:03:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:03:14.178333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c7c54-4538-44d7-a5c4-5ab46c1b740f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:14.179291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c7c54-4538-44d7-a5c4-5ab46c1b740f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:14.179565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c7c54-4538-44d7-a5c4-5ab46c1b740f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:14.207591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-880c7c54-4538-44d7-a5c4-5ab46c1b740f None None] MFA Rules not processed for user `de2fb55182604091b35826b2c2fd090c`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:03:14.208850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-880c7c54-4538-44d7-a5c4-5ab46c1b740f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:14.276025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1193/2382] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:03:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:03:14.280773 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd33b7f-7a6e-4e42-a230-1cd6ff29b0aa service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:14.290590 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd33b7f-7a6e-4e42-a230-1cd6ff29b0aa service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:14.315302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c9e29-47f6-48c9-a91b-76a4df018420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:14.315757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c9e29-47f6-48c9-a91b-76a4df018420 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:14.316036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c9e29-47f6-48c9-a91b-76a4df018420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:14.316378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c9e29-47f6-48c9-a91b-76a4df018420 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:14.317092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c9e29-47f6-48c9-a91b-76a4df018420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:14.324656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c9e29-47f6-48c9-a91b-76a4df018420 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:14.360737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1190/2383] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:03:14 2022] POST /identity/v3/projects => generated 381 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:03:14.364901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c7c54-4538-44d7-a5c4-5ab46c1b740f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:14.376738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c7c54-4538-44d7-a5c4-5ab46c1b740f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:14.382254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:03:14.440326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798982f-d1c8-41ee-b989-900db6695467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:14.441114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798982f-d1c8-41ee-b989-900db6695467 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:14.441393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798982f-d1c8-41ee-b989-900db6695467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:14.441697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798982f-d1c8-41ee-b989-900db6695467 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:14.442392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798982f-d1c8-41ee-b989-900db6695467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:14.444442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798982f-d1c8-41ee-b989-900db6695467 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:14.467093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1194/2384] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:03:14 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:03:14.471154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c9e29-47f6-48c9-a91b-76a4df018420 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:14.481789 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c9e29-47f6-48c9-a91b-76a4df018420 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:14.486454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafd74e-0758-4188-9357-d21238115917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:14.487134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafd74e-0758-4188-9357-d21238115917 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:14.487413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafd74e-0758-4188-9357-d21238115917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:14.487682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafd74e-0758-4188-9357-d21238115917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:14.488510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafd74e-0758-4188-9357-d21238115917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:14.490105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafd74e-0758-4188-9357-d21238115917 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:14.501533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1191/2385] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:03:14.506010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798982f-d1c8-41ee-b989-900db6695467 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:14.517194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798982f-d1c8-41ee-b989-900db6695467 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:14.522838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d4e81-3bc0-4c86-9d94-25e3b8faaf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:14.523766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d4e81-3bc0-4c86-9d94-25e3b8faaf81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:14.524045 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d4e81-3bc0-4c86-9d94-25e3b8faaf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:14.524358 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d4e81-3bc0-4c86-9d94-25e3b8faaf81 admin admin] PATH_INFO: `/v3/projects/8ff1a7d25fa04a8689523dbf80323d2d/users/7e04d5aa2b1f4e258c691918c587ebf8/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:14.541096 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d4e81-3bc0-4c86-9d94-25e3b8faaf81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff1a7d25fa04a8689523dbf80323d2d, user_id=7e04d5aa2b1f4e258c691918c587ebf8, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:14.543801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d4e81-3bc0-4c86-9d94-25e3b8faaf81 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:14.559805 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1195/2386] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:03:14 2022] PUT /identity/v3/projects/8ff1a7d25fa04a8689523dbf80323d2d/users/7e04d5aa2b1f4e258c691918c587ebf8/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:03:14.563798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafd74e-0758-4188-9357-d21238115917 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:14.575001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafd74e-0758-4188-9357-d21238115917 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:14.636370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63432af-6c18-4f61-891f-c8bae67675f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:14.637105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63432af-6c18-4f61-891f-c8bae67675f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:14.637392 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63432af-6c18-4f61-891f-c8bae67675f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:14.637669 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63432af-6c18-4f61-891f-c8bae67675f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:14.638413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63432af-6c18-4f61-891f-c8bae67675f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:14.640760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63432af-6c18-4f61-891f-c8bae67675f7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:14.651645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1192/2387] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:03:14.655762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d4e81-3bc0-4c86-9d94-25e3b8faaf81 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:14.666410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d4e81-3bc0-4c86-9d94-25e3b8faaf81 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:14.671517 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11282e1-70c4-4495-854c-f59fb5dcc904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:14.672355 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11282e1-70c4-4495-854c-f59fb5dcc904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:14.672643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11282e1-70c4-4495-854c-f59fb5dcc904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:14.672904 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11282e1-70c4-4495-854c-f59fb5dcc904 admin admin] PATH_INFO: `/v3/projects/8ff1a7d25fa04a8689523dbf80323d2d/users/7e04d5aa2b1f4e258c691918c587ebf8/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:14.674874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11282e1-70c4-4495-854c-f59fb5dcc904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff1a7d25fa04a8689523dbf80323d2d, user_id=7e04d5aa2b1f4e258c691918c587ebf8, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:14.677969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11282e1-70c4-4495-854c-f59fb5dcc904 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:14.689724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1196/2388] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:03:14 2022] PUT /identity/v3/projects/8ff1a7d25fa04a8689523dbf80323d2d/users/7e04d5aa2b1f4e258c691918c587ebf8/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:03:18.818366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63432af-6c18-4f61-891f-c8bae67675f7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:18.820649 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a63432af-6c18-4f61-891f-c8bae67675f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:18.921617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:03:18.928901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63432af-6c18-4f61-891f-c8bae67675f7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:19.015333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb7a3f-b951-4ee1-9669-b1aaa19c2771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:19.017212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb7a3f-b951-4ee1-9669-b1aaa19c2771 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:19.017788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb7a3f-b951-4ee1-9669-b1aaa19c2771 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:19.018217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb7a3f-b951-4ee1-9669-b1aaa19c2771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:19.033116 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-52bb7a3f-b951-4ee1-9669-b1aaa19c2771 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:19.065699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d5df-d40e-4c27-8cf4-2b454c452691 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:19.066239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d5df-d40e-4c27-8cf4-2b454c452691 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:19.066475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d5df-d40e-4c27-8cf4-2b454c452691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:19.097439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5d35d5df-d40e-4c27-8cf4-2b454c452691 None None] MFA Rules not processed for user `7e04d5aa2b1f4e258c691918c587ebf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:03:19.098664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5d35d5df-d40e-4c27-8cf4-2b454c452691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:19.124961 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb7a3f-b951-4ee1-9669-b1aaa19c2771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:19.126585 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb7a3f-b951-4ee1-9669-b1aaa19c2771 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:19.137780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1193/2389] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:03:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:03:19.177440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1197/2390] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:03:19.190066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb7a3f-b951-4ee1-9669-b1aaa19c2771 service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:19.191489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-52bb7a3f-b951-4ee1-9669-b1aaa19c2771 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:19.253371 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb7a3f-b951-4ee1-9669-b1aaa19c2771 service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:19.257078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d019688-b99a-429e-abc8-37ddf514a1ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097b12fbf6494336b11ece5c498a4103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:19.257750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d019688-b99a-429e-abc8-37ddf514a1ce service glance] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:19.258094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d019688-b99a-429e-abc8-37ddf514a1ce service glance] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:19.258372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d019688-b99a-429e-abc8-37ddf514a1ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:19.327189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:03:19.330008 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d019688-b99a-429e-abc8-37ddf514a1ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:19.331668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d019688-b99a-429e-abc8-37ddf514a1ce service glance] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:19.345690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1194/2391] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:03:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:03:19.490195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b49dc-2863-4a72-805b-e40ce42e1fc4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:19.490650 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b49dc-2863-4a72-805b-e40ce42e1fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:19.490925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b49dc-2863-4a72-805b-e40ce42e1fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:19.520929 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-759b49dc-2863-4a72-805b-e40ce42e1fc4 None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:03:19.522166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-759b49dc-2863-4a72-805b-e40ce42e1fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:19.593563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:03:19.614584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1198/2392] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:03:19.627332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d019688-b99a-429e-abc8-37ddf514a1ce service glance] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:19.628787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2d019688-b99a-429e-abc8-37ddf514a1ce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:19.708991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d019688-b99a-429e-abc8-37ddf514a1ce service glance] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:19.718291 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1576b87-3849-448a-9c83-005fc434f3e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:19.718917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576b87-3849-448a-9c83-005fc434f3e7 service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:19.720218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576b87-3849-448a-9c83-005fc434f3e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:19.720218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576b87-3849-448a-9c83-005fc434f3e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:19.749529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1576b87-3849-448a-9c83-005fc434f3e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:19.751156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1576b87-3849-448a-9c83-005fc434f3e7 service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:19.760020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1195/2393] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:03:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:03:28.509795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c0eb2-0d6b-4946-87dd-11676361212c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:28.510401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c0eb2-0d6b-4946-87dd-11676361212c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:28.510401 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c0eb2-0d6b-4946-87dd-11676361212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:28.529217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b24c0eb2-0d6b-4946-87dd-11676361212c None None] MFA Rules not processed for user `de2fb55182604091b35826b2c2fd090c`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:03:28.531630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b24c0eb2-0d6b-4946-87dd-11676361212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:28.539357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1199/2394] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:03:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:03:28.812983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1576b87-3849-448a-9c83-005fc434f3e7 service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:28.814688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b1576b87-3849-448a-9c83-005fc434f3e7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:28.875749 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1576b87-3849-448a-9c83-005fc434f3e7 service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:28.879654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77443dc5-51c2-402e-8b16-0686310903cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:28.880331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77443dc5-51c2-402e-8b16-0686310903cb service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:28.880612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77443dc5-51c2-402e-8b16-0686310903cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:28.880891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77443dc5-51c2-402e-8b16-0686310903cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:28.882741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-77443dc5-51c2-402e-8b16-0686310903cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:28.892434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77443dc5-51c2-402e-8b16-0686310903cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:28.893867 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77443dc5-51c2-402e-8b16-0686310903cb service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:28.901931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1196/2395] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:03:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:03:29.216336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c0eb2-0d6b-4946-87dd-11676361212c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:29.217960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b24c0eb2-0d6b-4946-87dd-11676361212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:29.301817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c0eb2-0d6b-4946-87dd-11676361212c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:29.307632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52a69a-14aa-464c-9c8f-4a17a9dc813f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:29.308340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52a69a-14aa-464c-9c8f-4a17a9dc813f service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:29.308625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52a69a-14aa-464c-9c8f-4a17a9dc813f service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:29.308903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52a69a-14aa-464c-9c8f-4a17a9dc813f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:29.310787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8b52a69a-14aa-464c-9c8f-4a17a9dc813f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:29.323438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52a69a-14aa-464c-9c8f-4a17a9dc813f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:29.325103 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52a69a-14aa-464c-9c8f-4a17a9dc813f service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:29.335452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1200/2396] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:03:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:03:30.746248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e6f3c-e301-4f4b-a067-2349a15a3001 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:30.746756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e6f3c-e301-4f4b-a067-2349a15a3001 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:30.746856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e6f3c-e301-4f4b-a067-2349a15a3001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:30.764421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a64e6f3c-e301-4f4b-a067-2349a15a3001 None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:03:30.765689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a64e6f3c-e301-4f4b-a067-2349a15a3001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:30.773336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1197/2397] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 21:03:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:03:30.786910 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52a69a-14aa-464c-9c8f-4a17a9dc813f service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:30.798607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52a69a-14aa-464c-9c8f-4a17a9dc813f service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:30.804731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e756d-4265-4b16-a531-e55307bc254b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:30.805465 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e756d-4265-4b16-a531-e55307bc254b service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:30.805822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e756d-4265-4b16-a531-e55307bc254b service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:30.806175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e756d-4265-4b16-a531-e55307bc254b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:30.815411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e756d-4265-4b16-a531-e55307bc254b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:30.817196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e756d-4265-4b16-a531-e55307bc254b service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:30.826668 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1201/2398] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:03:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:03:37.922995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e6f3c-e301-4f4b-a067-2349a15a3001 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:37.924663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a64e6f3c-e301-4f4b-a067-2349a15a3001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:37.941818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e6f3c-e301-4f4b-a067-2349a15a3001 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:37.947194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e401159-df51-4edf-93ea-584f5c892be4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:37.947847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e401159-df51-4edf-93ea-584f5c892be4 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:37.948154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e401159-df51-4edf-93ea-584f5c892be4 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:37.948435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e401159-df51-4edf-93ea-584f5c892be4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:37.950185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4e401159-df51-4edf-93ea-584f5c892be4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:38.014701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e401159-df51-4edf-93ea-584f5c892be4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:38.016500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e401159-df51-4edf-93ea-584f5c892be4 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:38.029013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1198/2399] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:03:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:03:40.823121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=45 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:03:45.904326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e756d-4265-4b16-a531-e55307bc254b service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:45.922852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e756d-4265-4b16-a531-e55307bc254b service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:45.939173 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263b3e4-1033-4c20-81e0-03c6d0f47bdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:45.941768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263b3e4-1033-4c20-81e0-03c6d0f47bdb service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:45.942143 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263b3e4-1033-4c20-81e0-03c6d0f47bdb service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:45.942488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263b3e4-1033-4c20-81e0-03c6d0f47bdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:45.946107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0263b3e4-1033-4c20-81e0-03c6d0f47bdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:45.963609 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263b3e4-1033-4c20-81e0-03c6d0f47bdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:45.966109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263b3e4-1033-4c20-81e0-03c6d0f47bdb service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:45.990422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1202/2400] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:03:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:03:48.022548 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:03:52.432656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579af62-9010-4078-be7a-f89f45db270e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:52.433318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579af62-9010-4078-be7a-f89f45db270e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:52.433318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579af62-9010-4078-be7a-f89f45db270e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:52.450605 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2579af62-9010-4078-be7a-f89f45db270e None None] MFA Rules not processed for user `aa7e6c5842a84e91ac94c1107ad29158`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:03:52.451796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2579af62-9010-4078-be7a-f89f45db270e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:52.459557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1199/2401] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:03:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:03:55.101527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263b3e4-1033-4c20-81e0-03c6d0f47bdb service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:03:55.116147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263b3e4-1033-4c20-81e0-03c6d0f47bdb service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:03:55.126201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68473d56-2581-4eff-8d7c-acb05098a4d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:03:55.127121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68473d56-2581-4eff-8d7c-acb05098a4d1 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:03:55.127407 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68473d56-2581-4eff-8d7c-acb05098a4d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:03:55.127655 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68473d56-2581-4eff-8d7c-acb05098a4d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:03:55.130034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-68473d56-2581-4eff-8d7c-acb05098a4d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:03:55.147788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68473d56-2581-4eff-8d7c-acb05098a4d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:03:55.149939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68473d56-2581-4eff-8d7c-acb05098a4d1 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:03:55.163747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1203/2402] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:03:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:04:02.445706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:04:05.165700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=12 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:04:05.348256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579af62-9010-4078-be7a-f89f45db270e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:04:05.359278 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579af62-9010-4078-be7a-f89f45db270e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:04:05.364264 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d983f5-8c77-40d7-9c07-4565df530f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:04:05.365001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d983f5-8c77-40d7-9c07-4565df530f21 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:04:05.365283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d983f5-8c77-40d7-9c07-4565df530f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:04:05.365551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d983f5-8c77-40d7-9c07-4565df530f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:04:05.375221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d983f5-8c77-40d7-9c07-4565df530f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:04:05.376678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d983f5-8c77-40d7-9c07-4565df530f21 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:04:05.386518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1200/2403] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:04:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:04:06.945237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68473d56-2581-4eff-8d7c-acb05098a4d1 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:04:06.957474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68473d56-2581-4eff-8d7c-acb05098a4d1 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:04:06.964868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dcf56-329a-4103-b794-f590c99e1cd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:04:06.965766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dcf56-329a-4103-b794-f590c99e1cd8 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:04:06.966137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dcf56-329a-4103-b794-f590c99e1cd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:04:06.966622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dcf56-329a-4103-b794-f590c99e1cd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:04:06.968590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-249dcf56-329a-4103-b794-f590c99e1cd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:04:06.982121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dcf56-329a-4103-b794-f590c99e1cd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:04:06.983816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dcf56-329a-4103-b794-f590c99e1cd8 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:04:07.004590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1204/2404] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:04:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:04:15.382415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=6 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:04:16.991114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:04:30.909187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aecb24-daf8-418c-bdfe-db815ef6ae23 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:04:30.909774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aecb24-daf8-418c-bdfe-db815ef6ae23 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:04:30.909774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aecb24-daf8-418c-bdfe-db815ef6ae23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:04:30.930652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a9aecb24-daf8-418c-bdfe-db815ef6ae23 None None] MFA Rules not processed for user `77cf43c378a9413dbd799a00bfa21eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:04:30.931907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a9aecb24-daf8-418c-bdfe-db815ef6ae23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:04:30.990310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1201/2405] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:04:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:04:31.424080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dcf56-329a-4103-b794-f590c99e1cd8 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:04:31.434510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dcf56-329a-4103-b794-f590c99e1cd8 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:04:31.438576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726ee51-e556-4c5a-902f-da5d93f90176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:04:31.439214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726ee51-e556-4c5a-902f-da5d93f90176 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:04:31.439492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726ee51-e556-4c5a-902f-da5d93f90176 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:04:31.439774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726ee51-e556-4c5a-902f-da5d93f90176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:04:31.441674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b726ee51-e556-4c5a-902f-da5d93f90176 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:04:31.451577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726ee51-e556-4c5a-902f-da5d93f90176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:04:31.453049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726ee51-e556-4c5a-902f-da5d93f90176 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:04:31.461461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1205/2406] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:04:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:04:40.983104 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=25 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:04:41.457546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:04:51.730610 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096c24f-a3c5-4d0f-9305-37ef6f224abf None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:04:51.731118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096c24f-a3c5-4d0f-9305-37ef6f224abf None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:04:51.731118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096c24f-a3c5-4d0f-9305-37ef6f224abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:04:51.750396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f096c24f-a3c5-4d0f-9305-37ef6f224abf None None] MFA Rules not processed for user `5f8c2e5e6c1b4833b0e9af0de8814d13`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:04:51.751729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f096c24f-a3c5-4d0f-9305-37ef6f224abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:04:51.811911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1202/2407] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:04:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:05:01.804055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=25 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:05:30.249605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726ee51-e556-4c5a-902f-da5d93f90176 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:30.272880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726ee51-e556-4c5a-902f-da5d93f90176 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:30.277298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc93e3a-8e67-4551-8e6f-545f043d709c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:30.278088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc93e3a-8e67-4551-8e6f-545f043d709c service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:30.278463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc93e3a-8e67-4551-8e6f-545f043d709c service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:30.278798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc93e3a-8e67-4551-8e6f-545f043d709c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:30.280846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5fc93e3a-8e67-4551-8e6f-545f043d709c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:30.403416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc93e3a-8e67-4551-8e6f-545f043d709c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:30.404895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc93e3a-8e67-4551-8e6f-545f043d709c service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:30.425216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1206/2408] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:05:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:05:30.696517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12991-dd77-4830-ae12-9641f9299184 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:30.697084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12991-dd77-4830-ae12-9641f9299184 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:30.697084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12991-dd77-4830-ae12-9641f9299184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:30.714907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c8a12991-dd77-4830-ae12-9641f9299184 None None] MFA Rules not processed for user `0965444614ae486e80e111f4126b52f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:05:30.716229 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c8a12991-dd77-4830-ae12-9641f9299184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:30.723809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1203/2409] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:05:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:05:34.390299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc93e3a-8e67-4551-8e6f-545f043d709c service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:34.391844 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5fc93e3a-8e67-4551-8e6f-545f043d709c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:34.445541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc93e3a-8e67-4551-8e6f-545f043d709c service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:34.451906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed1a1c-38bc-4012-81c0-41facae9856a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:34.452605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed1a1c-38bc-4012-81c0-41facae9856a service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:34.452870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed1a1c-38bc-4012-81c0-41facae9856a service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:34.453201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed1a1c-38bc-4012-81c0-41facae9856a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:34.457097 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-beed1a1c-38bc-4012-81c0-41facae9856a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:34.470586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed1a1c-38bc-4012-81c0-41facae9856a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:34.472740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed1a1c-38bc-4012-81c0-41facae9856a service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:34.489114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1207/2410] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:05:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:05:34.535501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12991-dd77-4830-ae12-9641f9299184 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:34.541207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c8a12991-dd77-4830-ae12-9641f9299184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:34.562156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12991-dd77-4830-ae12-9641f9299184 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:34.567724 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:34.568833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:34.569299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:34.569719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] PATH_INFO: `/v3/users/aa7e6c5842a84e91ac94c1107ad29158` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:34.571437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa7e6c5842a84e91ac94c1107ad29158)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:34.573435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:34.636871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa7e6c5842a84e91ac94c1107ad29158'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:05:34.637687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa7e6c5842a84e91ac94c1107ad29158'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:05:34.650347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa7e6c5842a84e91ac94c1107ad29158'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:05:34.651518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa7e6c5842a84e91ac94c1107ad29158'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:05:34.667450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-da2ef59b-5665-47eb-bc40-2fc817fd6744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa7e6c5842a84e91ac94c1107ad29158'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:05:34.685209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1204/2411] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:05:34 2022] DELETE /identity/v3/users/aa7e6c5842a84e91ac94c1107ad29158 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:05:34.887840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed1a1c-38bc-4012-81c0-41facae9856a service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:34.899525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed1a1c-38bc-4012-81c0-41facae9856a service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:34.964686 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:05:34.984050 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:34.985352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:34.985762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:34.986156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] PATH_INFO: `/v3/projects/c6f21a94f61e42c9b967e673f02eb270` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:34.987842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f21a94f61e42c9b967e673f02eb270)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:34.990216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:35.046237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f21a94f61e42c9b967e673f02eb270', 'request_id': 'req-3022f74a-4380-4b49-9b8b-f326cd6773da'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:05:35.056652 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f21a94f61e42c9b967e673f02eb270', 'request_id': 'req-3022f74a-4380-4b49-9b8b-f326cd6773da'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:05:35.067986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f21a94f61e42c9b967e673f02eb270', 'request_id': 'req-3022f74a-4380-4b49-9b8b-f326cd6773da'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:05:35.079990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] The token cache is being invalidate because project c6f21a94f61e42c9b967e673f02eb270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:05:35.080313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:05:35.080950 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3022f74a-4380-4b49-9b8b-f326cd6773da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:05:35.082516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1208/2412] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:05:34 2022] DELETE /identity/v3/projects/c6f21a94f61e42c9b967e673f02eb270 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:05:35.087545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9e0be-e78e-4445-911f-cb91a43d19ad None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:35.087941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9e0be-e78e-4445-911f-cb91a43d19ad None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:35.088266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9e0be-e78e-4445-911f-cb91a43d19ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:35.156295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-49e9e0be-e78e-4445-911f-cb91a43d19ad None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:05:35.157687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-49e9e0be-e78e-4445-911f-cb91a43d19ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:35.234350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1205/2413] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:05:35.244280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede9c6d-5afc-4cf4-8ebc-cf81a9534fe2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:35.244640 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede9c6d-5afc-4cf4-8ebc-cf81a9534fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:35.244983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede9c6d-5afc-4cf4-8ebc-cf81a9534fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:35.270303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2ede9c6d-5afc-4cf4-8ebc-cf81a9534fe2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:05:35.273373 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2ede9c6d-5afc-4cf4-8ebc-cf81a9534fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:35.285602 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1209/2414] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:05:35.287664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9e0be-e78e-4445-911f-cb91a43d19ad None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:35.300503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9e0be-e78e-4445-911f-cb91a43d19ad None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:35.306390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee6000-c44d-479f-9b94-270e956004cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:35.307081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee6000-c44d-479f-9b94-270e956004cd admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:35.308144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee6000-c44d-479f-9b94-270e956004cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:35.308423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee6000-c44d-479f-9b94-270e956004cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:35.309357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee6000-c44d-479f-9b94-270e956004cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:05:35.309820 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee6000-c44d-479f-9b94-270e956004cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:35.312086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee6000-c44d-479f-9b94-270e956004cd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:35.326975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1206/2415] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:05:35.331653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede9c6d-5afc-4cf4-8ebc-cf81a9534fe2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:35.343036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede9c6d-5afc-4cf4-8ebc-cf81a9534fe2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:35.348750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2462b6c-ccf4-4e5a-bd89-03336b285133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:35.349565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2462b6c-ccf4-4e5a-bd89-03336b285133 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:35.349998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2462b6c-ccf4-4e5a-bd89-03336b285133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:35.350363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2462b6c-ccf4-4e5a-bd89-03336b285133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:35.351269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2462b6c-ccf4-4e5a-bd89-03336b285133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:35.353355 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2462b6c-ccf4-4e5a-bd89-03336b285133 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:35.400167 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1210/2416] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:05:35 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:05:35.403320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee6000-c44d-479f-9b94-270e956004cd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:35.416480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee6000-c44d-479f-9b94-270e956004cd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:35.436866 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:05:35.482011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba5a33-4ea8-44fc-8510-3137e584635a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:35.482907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba5a33-4ea8-44fc-8510-3137e584635a admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:35.483364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba5a33-4ea8-44fc-8510-3137e584635a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:35.483751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba5a33-4ea8-44fc-8510-3137e584635a admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:35.484590 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba5a33-4ea8-44fc-8510-3137e584635a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:35.486753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba5a33-4ea8-44fc-8510-3137e584635a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:35.515377 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1207/2417] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:05:35 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:05:35.516738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2462b6c-ccf4-4e5a-bd89-03336b285133 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:35.528451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2462b6c-ccf4-4e5a-bd89-03336b285133 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:35.533349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a3258-bdf3-4479-8d2f-8c65d6dce48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:35.534073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a3258-bdf3-4479-8d2f-8c65d6dce48d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:35.534354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a3258-bdf3-4479-8d2f-8c65d6dce48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:35.534624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a3258-bdf3-4479-8d2f-8c65d6dce48d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:35.535350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a3258-bdf3-4479-8d2f-8c65d6dce48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:35.537018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a3258-bdf3-4479-8d2f-8c65d6dce48d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:35.549477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1211/2418] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:05:35.553435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba5a33-4ea8-44fc-8510-3137e584635a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:35.563470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba5a33-4ea8-44fc-8510-3137e584635a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:35.568429 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49007fd-764d-4c69-9599-cdcba221f3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:35.568982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49007fd-764d-4c69-9599-cdcba221f3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:35.569345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49007fd-764d-4c69-9599-cdcba221f3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:35.569538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49007fd-764d-4c69-9599-cdcba221f3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:35.570333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49007fd-764d-4c69-9599-cdcba221f3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:35.572001 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49007fd-764d-4c69-9599-cdcba221f3fb admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:35.582507 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1208/2419] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:05:35.586896 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a3258-bdf3-4479-8d2f-8c65d6dce48d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:35.597760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a3258-bdf3-4479-8d2f-8c65d6dce48d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:35.602425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87086ecb-5497-42f2-8e20-013f72e8c8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:35.603323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086ecb-5497-42f2-8e20-013f72e8c8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:35.603600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086ecb-5497-42f2-8e20-013f72e8c8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:35.603869 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086ecb-5497-42f2-8e20-013f72e8c8b3 admin admin] PATH_INFO: `/v3/projects/b85a8791815f40f98ed1e81e58b01fd6/users/68f6997db7c141529c37cc64955df779/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:35.620040 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87086ecb-5497-42f2-8e20-013f72e8c8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b85a8791815f40f98ed1e81e58b01fd6, user_id=68f6997db7c141529c37cc64955df779, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:35.622788 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87086ecb-5497-42f2-8e20-013f72e8c8b3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:35.639582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1212/2420] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:05:35 2022] PUT /identity/v3/projects/b85a8791815f40f98ed1e81e58b01fd6/users/68f6997db7c141529c37cc64955df779/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:05:35.645839 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d046b8-91db-4342-ae34-88dc2473270d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:35.646211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d046b8-91db-4342-ae34-88dc2473270d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:35.646412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d046b8-91db-4342-ae34-88dc2473270d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:35.667939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-57d046b8-91db-4342-ae34-88dc2473270d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:05:35.669423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-57d046b8-91db-4342-ae34-88dc2473270d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:35.750305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1209/2421] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:05:35.760848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87086ecb-5497-42f2-8e20-013f72e8c8b3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:35.762679 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-87086ecb-5497-42f2-8e20-013f72e8c8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:35.838474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87086ecb-5497-42f2-8e20-013f72e8c8b3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:35.842600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bdf01-ff7c-4de7-a248-5d6173405d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:35.843292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bdf01-ff7c-4de7-a248-5d6173405d6c service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:35.843623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bdf01-ff7c-4de7-a248-5d6173405d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:35.843905 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bdf01-ff7c-4de7-a248-5d6173405d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:35.855370 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468bdf01-ff7c-4de7-a248-5d6173405d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:35.858756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468bdf01-ff7c-4de7-a248-5d6173405d6c service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:35.872390 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1213/2422] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:05:40.279284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23056b-f377-4990-94d3-56d08de3d6d0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:40.280242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23056b-f377-4990-94d3-56d08de3d6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:40.280630 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23056b-f377-4990-94d3-56d08de3d6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:40.315506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8a23056b-f377-4990-94d3-56d08de3d6d0 None None] MFA Rules not processed for user `68f6997db7c141529c37cc64955df779`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:05:40.317073 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8a23056b-f377-4990-94d3-56d08de3d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:40.416479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1210/2423] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:05:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:05:40.429117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bdf01-ff7c-4de7-a248-5d6173405d6c service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:05:40.430883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-468bdf01-ff7c-4de7-a248-5d6173405d6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:40.451192 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:05:40.543881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bdf01-ff7c-4de7-a248-5d6173405d6c service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:05:40.549975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24020090-3243-4f69-a80d-13dc03ee2074 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:05:40.550671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24020090-3243-4f69-a80d-13dc03ee2074 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:40.550954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24020090-3243-4f69-a80d-13dc03ee2074 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:40.551231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24020090-3243-4f69-a80d-13dc03ee2074 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:40.560086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24020090-3243-4f69-a80d-13dc03ee2074 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:05:40.561730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24020090-3243-4f69-a80d-13dc03ee2074 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:05:40.571156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1214/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:05:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:05:42.181746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b38c0-30b2-48f5-86f3-eca93be33b1b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:05:42.182844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b38c0-30b2-48f5-86f3-eca93be33b1b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:05:42.183283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b38c0-30b2-48f5-86f3-eca93be33b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:05:42.204705 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-672b38c0-30b2-48f5-86f3-eca93be33b1b None None] MFA Rules not processed for user `68f6997db7c141529c37cc64955df779`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:05:42.206791 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-672b38c0-30b2-48f5-86f3-eca93be33b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:05:42.222910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1211/2425] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:05:50.567041 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=40 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:05:52.197559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:06:30.831721 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24020090-3243-4f69-a80d-13dc03ee2074 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:06:30.833333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-24020090-3243-4f69-a80d-13dc03ee2074 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:06:30.913767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24020090-3243-4f69-a80d-13dc03ee2074 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:06:30.919678 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1b8fb-71a6-43fb-898e-e09be72fc852 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:06:30.920385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1b8fb-71a6-43fb-898e-e09be72fc852 service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:06:30.920669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1b8fb-71a6-43fb-898e-e09be72fc852 service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:06:30.920951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1b8fb-71a6-43fb-898e-e09be72fc852 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:06:30.923339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-feb1b8fb-71a6-43fb-898e-e09be72fc852 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:06:30.935094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1b8fb-71a6-43fb-898e-e09be72fc852 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:06:30.936737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1b8fb-71a6-43fb-898e-e09be72fc852 service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:06:30.946866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1215/2426] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:06:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:06:40.942778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=32 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:06:41.379828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953f51c-8acd-414c-a1d9-d6749912b60d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:06:41.380356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953f51c-8acd-414c-a1d9-d6749912b60d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:06:41.380451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953f51c-8acd-414c-a1d9-d6749912b60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:06:41.409748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7953f51c-8acd-414c-a1d9-d6749912b60d None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:06:41.410931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7953f51c-8acd-414c-a1d9-d6749912b60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:06:41.468847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1212/2427] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 21:06:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:06:41.508984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1b8fb-71a6-43fb-898e-e09be72fc852 service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:06:41.510674 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-feb1b8fb-71a6-43fb-898e-e09be72fc852 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:06:41.585123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1b8fb-71a6-43fb-898e-e09be72fc852 service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:06:41.591132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ed307-aceb-4b37-9b81-b7dec501e6f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:06:41.591793 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ed307-aceb-4b37-9b81-b7dec501e6f7 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:06:41.592074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ed307-aceb-4b37-9b81-b7dec501e6f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:06:41.592376 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ed307-aceb-4b37-9b81-b7dec501e6f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:06:41.602118 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ed307-aceb-4b37-9b81-b7dec501e6f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:06:41.603727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ed307-aceb-4b37-9b81-b7dec501e6f7 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:06:41.612702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1216/2428] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:06:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:06:51.461945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=29 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:06:51.608744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=32 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:06:58.248130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fa611-5365-49a8-bdf5-ab4bfab78e20 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:06:58.248580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fa611-5365-49a8-bdf5-ab4bfab78e20 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:06:58.248685 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fa611-5365-49a8-bdf5-ab4bfab78e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:06:58.266901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5e6fa611-5365-49a8-bdf5-ab4bfab78e20 None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:06:58.268133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5e6fa611-5365-49a8-bdf5-ab4bfab78e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:06:58.276164 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1213/2429] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:06:58.288161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ed307-aceb-4b37-9b81-b7dec501e6f7 service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:06:58.299158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ed307-aceb-4b37-9b81-b7dec501e6f7 service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:06:58.304566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf34ec-aa1b-4c99-a5c7-177f0341dd79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:06:58.305287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf34ec-aa1b-4c99-a5c7-177f0341dd79 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:06:58.305568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf34ec-aa1b-4c99-a5c7-177f0341dd79 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:06:58.305845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf34ec-aa1b-4c99-a5c7-177f0341dd79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:06:58.314034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf34ec-aa1b-4c99-a5c7-177f0341dd79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:06:58.315649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf34ec-aa1b-4c99-a5c7-177f0341dd79 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:06:58.324016 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1217/2430] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:06:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:07:08.261787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:07:08.320491 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:07:12.165114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fa611-5365-49a8-bdf5-ab4bfab78e20 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:12.167835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5e6fa611-5365-49a8-bdf5-ab4bfab78e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:12.260997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fa611-5365-49a8-bdf5-ab4bfab78e20 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:12.264962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df27be-4746-49f4-b6d0-7fdf3b5150cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:12.265617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df27be-4746-49f4-b6d0-7fdf3b5150cf service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:12.265901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df27be-4746-49f4-b6d0-7fdf3b5150cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:12.266185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df27be-4746-49f4-b6d0-7fdf3b5150cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:12.268011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-62df27be-4746-49f4-b6d0-7fdf3b5150cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:12.330287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df27be-4746-49f4-b6d0-7fdf3b5150cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:12.331732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df27be-4746-49f4-b6d0-7fdf3b5150cf service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:12.341144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1214/2431] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:07:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:07:20.233825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf34ec-aa1b-4c99-a5c7-177f0341dd79 service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:20.254688 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf34ec-aa1b-4c99-a5c7-177f0341dd79 service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:20.258809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7978e-1c62-4422-af21-dd07c7dbfc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:20.259452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7978e-1c62-4422-af21-dd07c7dbfc7c service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:20.259729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7978e-1c62-4422-af21-dd07c7dbfc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:20.259997 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7978e-1c62-4422-af21-dd07c7dbfc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:20.262087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f6a7978e-1c62-4422-af21-dd07c7dbfc7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:20.276372 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7978e-1c62-4422-af21-dd07c7dbfc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:20.277798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7978e-1c62-4422-af21-dd07c7dbfc7c service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:20.288813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1218/2432] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:07:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:07:21.478769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df27be-4746-49f4-b6d0-7fdf3b5150cf service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:21.480859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-62df27be-4746-49f4-b6d0-7fdf3b5150cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:21.513972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df27be-4746-49f4-b6d0-7fdf3b5150cf service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:21.518767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2eb90-3b56-402b-8be5-9e35f21047b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:21.519421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2eb90-3b56-402b-8be5-9e35f21047b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:21.519709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2eb90-3b56-402b-8be5-9e35f21047b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:21.519988 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2eb90-3b56-402b-8be5-9e35f21047b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:21.523854 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cff2eb90-3b56-402b-8be5-9e35f21047b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:21.550505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2eb90-3b56-402b-8be5-9e35f21047b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:21.552894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2eb90-3b56-402b-8be5-9e35f21047b7 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:21.569139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1215/2433] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:07:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:07:22.656868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058620d-c368-4ce5-8736-6e56d75ef2d9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:22.657466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058620d-c368-4ce5-8736-6e56d75ef2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:22.657576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058620d-c368-4ce5-8736-6e56d75ef2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:22.681242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e058620d-c368-4ce5-8736-6e56d75ef2d9 None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:07:22.682477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e058620d-c368-4ce5-8736-6e56d75ef2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:22.691634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1219/2434] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jul 11 21:07:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:07:22.710616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2eb90-3b56-402b-8be5-9e35f21047b7 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:22.725045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2eb90-3b56-402b-8be5-9e35f21047b7 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:22.730259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6f5fa-de04-46ae-bad2-01b5355327fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:22.730997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6f5fa-de04-46ae-bad2-01b5355327fb service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:22.731351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6f5fa-de04-46ae-bad2-01b5355327fb service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:22.731696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6f5fa-de04-46ae-bad2-01b5355327fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:22.741889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6f5fa-de04-46ae-bad2-01b5355327fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:22.743765 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6f5fa-de04-46ae-bad2-01b5355327fb service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:22.771354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1216/2435] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:07:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:07:24.482259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1240d1a-27a1-4410-acd6-6ef85f41eeda None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:24.482720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1240d1a-27a1-4410-acd6-6ef85f41eeda None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:24.482810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1240d1a-27a1-4410-acd6-6ef85f41eeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:24.501034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c1240d1a-27a1-4410-acd6-6ef85f41eeda None None] MFA Rules not processed for user `5f8c2e5e6c1b4833b0e9af0de8814d13`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:07:24.502240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c1240d1a-27a1-4410-acd6-6ef85f41eeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:24.509600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1220/2436] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:07:32.141464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ccac-41f3-4afd-b491-e9dfe9ccdf1f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:32.141952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ccac-41f3-4afd-b491-e9dfe9ccdf1f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:32.142049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ccac-41f3-4afd-b491-e9dfe9ccdf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:32.161105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8ac3ccac-41f3-4afd-b491-e9dfe9ccdf1f None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:07:32.162357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8ac3ccac-41f3-4afd-b491-e9dfe9ccdf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:32.169824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1217/2437] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jul 11 21:07:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:07:32.182419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1240d1a-27a1-4410-acd6-6ef85f41eeda None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:32.194593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1240d1a-27a1-4410-acd6-6ef85f41eeda None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:32.199893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce90635-f94b-4912-a2b8-287926f357a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:32.200588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce90635-f94b-4912-a2b8-287926f357a1 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:32.201005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce90635-f94b-4912-a2b8-287926f357a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:32.201283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce90635-f94b-4912-a2b8-287926f357a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:32.211622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce90635-f94b-4912-a2b8-287926f357a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:32.213754 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce90635-f94b-4912-a2b8-287926f357a1 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:32.222947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1221/2438] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:07:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:07:37.139114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd6a5-b132-436d-99a3-01184bb1beaa None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:37.139704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd6a5-b132-436d-99a3-01184bb1beaa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:37.139808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd6a5-b132-436d-99a3-01184bb1beaa None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:37.141315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1218/2439] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jul 11 21:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 21:07:37.152086 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b16ae-5f60-4d2d-8d68-5429d050f328 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:37.152965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b16ae-5f60-4d2d-8d68-5429d050f328 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:37.153250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b16ae-5f60-4d2d-8d68-5429d050f328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:37.193419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-904b16ae-5f60-4d2d-8d68-5429d050f328 None None] MFA Rules not processed for user `946d11b552754364b55eb495f68ab852`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:07:37.194656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-904b16ae-5f60-4d2d-8d68-5429d050f328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:37.204094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1222/2440] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jul 11 21:07:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:07:37.219646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd6a5-b132-436d-99a3-01184bb1beaa None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:37.233921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd6a5-b132-436d-99a3-01184bb1beaa None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:37.239691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b641652-4d46-4f25-af91-1c0e6e024f86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:37.240379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b641652-4d46-4f25-af91-1c0e6e024f86 service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:37.240661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b641652-4d46-4f25-af91-1c0e6e024f86 service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:37.240938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b641652-4d46-4f25-af91-1c0e6e024f86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:37.249875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b641652-4d46-4f25-af91-1c0e6e024f86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:37.251480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b641652-4d46-4f25-af91-1c0e6e024f86 service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:37.262236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1219/2441] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:07:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:07:40.594526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b16ae-5f60-4d2d-8d68-5429d050f328 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:40.596653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-904b16ae-5f60-4d2d-8d68-5429d050f328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:40.614216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b16ae-5f60-4d2d-8d68-5429d050f328 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:40.620101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:40.621038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:40.621323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:40.621888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] PATH_INFO: `/v3/users/de2fb55182604091b35826b2c2fd090c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:40.623129 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2fb55182604091b35826b2c2fd090c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:40.625180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:40.677848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2fb55182604091b35826b2c2fd090c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:40.684699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2fb55182604091b35826b2c2fd090c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:40.685395 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2fb55182604091b35826b2c2fd090c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:40.694088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2fb55182604091b35826b2c2fd090c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:40.709843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2fb55182604091b35826b2c2fd090c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:40.716150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1223/2442] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:07:40 2022] DELETE /identity/v3/users/de2fb55182604091b35826b2c2fd090c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:07:40.923456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b641652-4d46-4f25-af91-1c0e6e024f86 service placement] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:40.929062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=84 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:07:40.938965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b641652-4d46-4f25-af91-1c0e6e024f86 service placement] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:40.999391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:41.000328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:41.000609 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:41.000875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] PATH_INFO: `/v3/projects/6f54283593cd4e71ba6b700377bf0b4e` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:41.002281 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f54283593cd4e71ba6b700377bf0b4e)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:41.004142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:41.053817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f54283593cd4e71ba6b700377bf0b4e', 'request_id': 'req-3114baef-b6a6-49da-b2be-007c58bca094'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.060404 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f54283593cd4e71ba6b700377bf0b4e', 'request_id': 'req-3114baef-b6a6-49da-b2be-007c58bca094'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.068484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f54283593cd4e71ba6b700377bf0b4e', 'request_id': 'req-3114baef-b6a6-49da-b2be-007c58bca094'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.079061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] The token cache is being invalidate because project 6f54283593cd4e71ba6b700377bf0b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:07:41.079345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.079945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.081537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1220/2443] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:07:40 2022] DELETE /identity/v3/projects/6f54283593cd4e71ba6b700377bf0b4e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:07:41.085600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:41.087036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:41.100334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f024dc-aec3-40d4-96f4-46f0c89e0489 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:41.162599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:41.163943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:41.164382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:41.164890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] PATH_INFO: `/v3/users/7e04d5aa2b1f4e258c691918c587ebf8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:41.166592 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e04d5aa2b1f4e258c691918c587ebf8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:41.169065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:41.174855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:07:41.224816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e04d5aa2b1f4e258c691918c587ebf8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.232270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e04d5aa2b1f4e258c691918c587ebf8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.233103 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e04d5aa2b1f4e258c691918c587ebf8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.241036 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e04d5aa2b1f4e258c691918c587ebf8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.256175 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-45ddd0cf-d3b5-480c-b46d-7ec490c88d88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e04d5aa2b1f4e258c691918c587ebf8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.263380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1224/2444] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:07:41 2022] DELETE /identity/v3/users/7e04d5aa2b1f4e258c691918c587ebf8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:07:41.451127 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:41.452625 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:41.466954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114baef-b6a6-49da-b2be-007c58bca094 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:41.527753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:41.528666 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:41.528942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:41.529210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] PATH_INFO: `/v3/projects/8ff1a7d25fa04a8689523dbf80323d2d` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:41.530406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff1a7d25fa04a8689523dbf80323d2d)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:41.532341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:41.580758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:07:41.597483 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff1a7d25fa04a8689523dbf80323d2d', 'request_id': 'req-3df2419c-fbe6-46d4-8326-b961d7bdcf50'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.603916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff1a7d25fa04a8689523dbf80323d2d', 'request_id': 'req-3df2419c-fbe6-46d4-8326-b961d7bdcf50'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.613624 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff1a7d25fa04a8689523dbf80323d2d', 'request_id': 'req-3df2419c-fbe6-46d4-8326-b961d7bdcf50'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.624694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] The token cache is being invalidate because project 8ff1a7d25fa04a8689523dbf80323d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:07:41.625015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.625620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3df2419c-fbe6-46d4-8326-b961d7bdcf50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:07:41.627249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1221/2445] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:07:41 2022] DELETE /identity/v3/projects/8ff1a7d25fa04a8689523dbf80323d2d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:07:41.632575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6378275-c879-4c67-b7ce-c21f36598e7c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:41.633053 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6378275-c879-4c67-b7ce-c21f36598e7c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:41.633149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6378275-c879-4c67-b7ce-c21f36598e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:41.651730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b6378275-c879-4c67-b7ce-c21f36598e7c None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:07:41.652995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b6378275-c879-4c67-b7ce-c21f36598e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:41.717430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1225/2446] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:07:41.727974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf0d57-8a53-49a8-bb21-e7198514c8a2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:41.728170 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf0d57-8a53-49a8-bb21-e7198514c8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:41.728451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf0d57-8a53-49a8-bb21-e7198514c8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:41.748346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0dbf0d57-8a53-49a8-bb21-e7198514c8a2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:07:41.749596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0dbf0d57-8a53-49a8-bb21-e7198514c8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:41.758978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1222/2447] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:07:41.763519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6378275-c879-4c67-b7ce-c21f36598e7c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:41.774424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6378275-c879-4c67-b7ce-c21f36598e7c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:41.779740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dff40-6420-4f46-967d-dec4378ffa52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:41.780437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dff40-6420-4f46-967d-dec4378ffa52 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:41.780724 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dff40-6420-4f46-967d-dec4378ffa52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:41.781111 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dff40-6420-4f46-967d-dec4378ffa52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:41.781851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dff40-6420-4f46-967d-dec4378ffa52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:07:41.782152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dff40-6420-4f46-967d-dec4378ffa52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:41.783779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dff40-6420-4f46-967d-dec4378ffa52 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:41.799807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1226/2448] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:07:41.801157 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf0d57-8a53-49a8-bb21-e7198514c8a2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:41.810921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf0d57-8a53-49a8-bb21-e7198514c8a2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:41.819344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f605a-f2ff-4a9a-990b-7b483477a48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:41.819344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f605a-f2ff-4a9a-990b-7b483477a48c admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:41.819344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f605a-f2ff-4a9a-990b-7b483477a48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:41.819344 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f605a-f2ff-4a9a-990b-7b483477a48c admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:41.819782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f605a-f2ff-4a9a-990b-7b483477a48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:41.820603 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f605a-f2ff-4a9a-990b-7b483477a48c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:41.862648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1223/2449] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:07:41 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:07:41.864405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dff40-6420-4f46-967d-dec4378ffa52 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:41.875909 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dff40-6420-4f46-967d-dec4378ffa52 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:41.945382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b65746-cc50-4796-ada0-b26a3d0ee8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:41.946078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b65746-cc50-4796-ada0-b26a3d0ee8e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:41.946354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b65746-cc50-4796-ada0-b26a3d0ee8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:41.946659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b65746-cc50-4796-ada0-b26a3d0ee8e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:41.947363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b65746-cc50-4796-ada0-b26a3d0ee8e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:41.949336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b65746-cc50-4796-ada0-b26a3d0ee8e5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:41.970777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:07:41.981784 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1227/2450] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:07:41 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:07:41.983439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f605a-f2ff-4a9a-990b-7b483477a48c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:41.998774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f605a-f2ff-4a9a-990b-7b483477a48c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:42.009757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2cb51-258e-4e60-9c59-163b0f2d385d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:42.010861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2cb51-258e-4e60-9c59-163b0f2d385d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:42.011148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2cb51-258e-4e60-9c59-163b0f2d385d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:42.011426 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2cb51-258e-4e60-9c59-163b0f2d385d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:42.012962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2cb51-258e-4e60-9c59-163b0f2d385d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:42.014600 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2cb51-258e-4e60-9c59-163b0f2d385d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:42.027922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1224/2451] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:07:42.032612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b65746-cc50-4796-ada0-b26a3d0ee8e5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:42.050183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b65746-cc50-4796-ada0-b26a3d0ee8e5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:42.056707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9b065-1a91-49d3-a898-16186726ff82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:42.057482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9b065-1a91-49d3-a898-16186726ff82 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:42.057767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9b065-1a91-49d3-a898-16186726ff82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:42.058039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9b065-1a91-49d3-a898-16186726ff82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:42.058775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9b065-1a91-49d3-a898-16186726ff82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:42.060451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9b065-1a91-49d3-a898-16186726ff82 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:42.071506 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1228/2452] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:07:42.075750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2cb51-258e-4e60-9c59-163b0f2d385d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:42.088351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2cb51-258e-4e60-9c59-163b0f2d385d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:42.093189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf8799-082f-4b7a-b89b-ea13e6af24c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:42.094110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8799-082f-4b7a-b89b-ea13e6af24c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:42.094435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8799-082f-4b7a-b89b-ea13e6af24c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:42.094596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8799-082f-4b7a-b89b-ea13e6af24c4 admin admin] PATH_INFO: `/v3/projects/3a26558aef284a9cbb955c7cdaa58e4e/users/e3223a1d1dc1445c8b4aa118eb9747fd/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:42.109882 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf8799-082f-4b7a-b89b-ea13e6af24c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a26558aef284a9cbb955c7cdaa58e4e, user_id=e3223a1d1dc1445c8b4aa118eb9747fd, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:42.112550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf8799-082f-4b7a-b89b-ea13e6af24c4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:42.128349 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1225/2453] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:07:42 2022] PUT /identity/v3/projects/3a26558aef284a9cbb955c7cdaa58e4e/users/e3223a1d1dc1445c8b4aa118eb9747fd/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:07:42.133608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283f861-fbc2-4aed-8495-216d3d27605d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:42.133902 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283f861-fbc2-4aed-8495-216d3d27605d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:42.134181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283f861-fbc2-4aed-8495-216d3d27605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:42.152474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b283f861-fbc2-4aed-8495-216d3d27605d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:07:42.153696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b283f861-fbc2-4aed-8495-216d3d27605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:42.215733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1229/2454] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:07:42.225600 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf8799-082f-4b7a-b89b-ea13e6af24c4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:42.227566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-14bf8799-082f-4b7a-b89b-ea13e6af24c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:42.285533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf8799-082f-4b7a-b89b-ea13e6af24c4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:42.289628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fb946-bc1f-4dc5-8537-5958f1e3da17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:42.290279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb946-bc1f-4dc5-8537-5958f1e3da17 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:42.290556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb946-bc1f-4dc5-8537-5958f1e3da17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:42.290833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb946-bc1f-4dc5-8537-5958f1e3da17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:42.301016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fb946-bc1f-4dc5-8537-5958f1e3da17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:42.302464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fb946-bc1f-4dc5-8537-5958f1e3da17 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:42.312140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1226/2455] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:07:46.654070 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83bcfe-6c6c-42dd-b3d5-3d394c2df2ad None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:46.654579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83bcfe-6c6c-42dd-b3d5-3d394c2df2ad None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:46.654579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83bcfe-6c6c-42dd-b3d5-3d394c2df2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:46.695444 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9b83bcfe-6c6c-42dd-b3d5-3d394c2df2ad None None] MFA Rules not processed for user `e3223a1d1dc1445c8b4aa118eb9747fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:07:46.696827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9b83bcfe-6c6c-42dd-b3d5-3d394c2df2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:46.771056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1230/2456] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:07:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:07:46.775557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fb946-bc1f-4dc5-8537-5958f1e3da17 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:46.785606 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fb946-bc1f-4dc5-8537-5958f1e3da17 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:46.790300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcb622-2750-498b-99c9-eacc37e9252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:46.790974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcb622-2750-498b-99c9-eacc37e9252b admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:46.791280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcb622-2750-498b-99c9-eacc37e9252b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:46.791550 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcb622-2750-498b-99c9-eacc37e9252b admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:46.792287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcb622-2750-498b-99c9-eacc37e9252b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:46.794206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcb622-2750-498b-99c9-eacc37e9252b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:46.826885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1227/2457] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:07:46 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:07:46.831157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83bcfe-6c6c-42dd-b3d5-3d394c2df2ad None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:46.842994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83bcfe-6c6c-42dd-b3d5-3d394c2df2ad None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:46.907412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55c1d8-d7c8-4f92-a5c1-d69cac37abff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:46.908128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55c1d8-d7c8-4f92-a5c1-d69cac37abff admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:46.908438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55c1d8-d7c8-4f92-a5c1-d69cac37abff admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:46.908716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55c1d8-d7c8-4f92-a5c1-d69cac37abff admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:46.909464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55c1d8-d7c8-4f92-a5c1-d69cac37abff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:46.912180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55c1d8-d7c8-4f92-a5c1-d69cac37abff admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:46.940301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1231/2458] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:07:46 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:07:46.940782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcb622-2750-498b-99c9-eacc37e9252b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:46.951692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcb622-2750-498b-99c9-eacc37e9252b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:46.961735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77798d9-2266-477d-bb56-75398188379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:46.962694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77798d9-2266-477d-bb56-75398188379d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:46.962992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77798d9-2266-477d-bb56-75398188379d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:46.963498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77798d9-2266-477d-bb56-75398188379d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:46.964423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77798d9-2266-477d-bb56-75398188379d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:46.966111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77798d9-2266-477d-bb56-75398188379d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:46.982683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1228/2459] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:07:46.983330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55c1d8-d7c8-4f92-a5c1-d69cac37abff admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:46.994033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55c1d8-d7c8-4f92-a5c1-d69cac37abff admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:46.999239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3bfe7-65a8-48d7-82f6-e91192529384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:47.000312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3bfe7-65a8-48d7-82f6-e91192529384 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:47.000712 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3bfe7-65a8-48d7-82f6-e91192529384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:47.001083 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3bfe7-65a8-48d7-82f6-e91192529384 admin admin] PATH_INFO: `/v3/projects/dfabd2f957cf47598ef20e4293b87e14/users/0862fcd84e21434389874fab02cd92e3/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:47.004857 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:07:47.020343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3bfe7-65a8-48d7-82f6-e91192529384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfabd2f957cf47598ef20e4293b87e14, user_id=0862fcd84e21434389874fab02cd92e3, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:47.023166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3bfe7-65a8-48d7-82f6-e91192529384 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:47.042931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1232/2460] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:07:46 2022] PUT /identity/v3/projects/dfabd2f957cf47598ef20e4293b87e14/users/0862fcd84e21434389874fab02cd92e3/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:07:47.047352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77798d9-2266-477d-bb56-75398188379d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:47.077324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77798d9-2266-477d-bb56-75398188379d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:47.102909 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:07:47.157983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b85d5-3096-48e0-b190-fac683fdca59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:47.159120 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b85d5-3096-48e0-b190-fac683fdca59 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:47.159588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b85d5-3096-48e0-b190-fac683fdca59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:47.160046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b85d5-3096-48e0-b190-fac683fdca59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:47.161470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b85d5-3096-48e0-b190-fac683fdca59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:47.163332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b85d5-3096-48e0-b190-fac683fdca59 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:47.175619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1229/2461] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:07:47.180098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3bfe7-65a8-48d7-82f6-e91192529384 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:47.191211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3bfe7-65a8-48d7-82f6-e91192529384 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:47.196935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8595c8-4e53-4ff8-bd89-9eeb6c3899e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:47.197950 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8595c8-4e53-4ff8-bd89-9eeb6c3899e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:47.198135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8595c8-4e53-4ff8-bd89-9eeb6c3899e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:47.198409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8595c8-4e53-4ff8-bd89-9eeb6c3899e8 admin admin] PATH_INFO: `/v3/projects/dfabd2f957cf47598ef20e4293b87e14/users/0862fcd84e21434389874fab02cd92e3/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:47.201758 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8595c8-4e53-4ff8-bd89-9eeb6c3899e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfabd2f957cf47598ef20e4293b87e14, user_id=0862fcd84e21434389874fab02cd92e3, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:47.204771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8595c8-4e53-4ff8-bd89-9eeb6c3899e8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:47.221600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1233/2462] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:07:47 2022] PUT /identity/v3/projects/dfabd2f957cf47598ef20e4293b87e14/users/0862fcd84e21434389874fab02cd92e3/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:07:48.841954 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b85d5-3096-48e0-b190-fac683fdca59 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:48.907328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b85d5-3096-48e0-b190-fac683fdca59 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:48.987284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5a949-bed7-4133-9b44-35c79279898c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:48.988336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a949-bed7-4133-9b44-35c79279898c service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:48.990169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a949-bed7-4133-9b44-35c79279898c service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:48.990662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a949-bed7-4133-9b44-35c79279898c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:48.992944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b2a5a949-bed7-4133-9b44-35c79279898c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:49.089700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5a949-bed7-4133-9b44-35c79279898c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:49.091232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5a949-bed7-4133-9b44-35c79279898c service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:49.116681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1230/2463] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:07:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:07:52.941302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7371d1c-160f-4784-9408-b8fcf5f6cbf0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:52.943195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7371d1c-160f-4784-9408-b8fcf5f6cbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:52.943195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7371d1c-160f-4784-9408-b8fcf5f6cbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:52.995467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c7371d1c-160f-4784-9408-b8fcf5f6cbf0 None None] MFA Rules not processed for user `0862fcd84e21434389874fab02cd92e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:07:52.996940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c7371d1c-160f-4784-9408-b8fcf5f6cbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:53.147044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1234/2464] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:07:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:07:53.161901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5a949-bed7-4133-9b44-35c79279898c service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:07:53.164012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b2a5a949-bed7-4133-9b44-35c79279898c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:53.216427 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:07:53.307845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5a949-bed7-4133-9b44-35c79279898c service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:07:53.315180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d95075-6941-4d99-b07f-6b07804b12dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:07:53.316237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d95075-6941-4d99-b07f-6b07804b12dd service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:53.317145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d95075-6941-4d99-b07f-6b07804b12dd service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:53.317661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d95075-6941-4d99-b07f-6b07804b12dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:53.422513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d95075-6941-4d99-b07f-6b07804b12dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:07:53.425844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d95075-6941-4d99-b07f-6b07804b12dd service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:07:53.460676 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1231/2465] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:07:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:07:54.710798 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06135a5c-0760-41f6-aa1f-e5ecbbf9ceee None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:07:54.711289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06135a5c-0760-41f6-aa1f-e5ecbbf9ceee None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:07:54.711930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06135a5c-0760-41f6-aa1f-e5ecbbf9ceee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:07:54.732778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-06135a5c-0760-41f6-aa1f-e5ecbbf9ceee None None] MFA Rules not processed for user `e3223a1d1dc1445c8b4aa118eb9747fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:07:54.733975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-06135a5c-0760-41f6-aa1f-e5ecbbf9ceee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:07:54.741912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1235/2466] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:07:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:03.433987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=46 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:04.727190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:11.497707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2e731-7c7e-489e-a08f-6cfd5e7d1b9d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:11.498261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2e731-7c7e-489e-a08f-6cfd5e7d1b9d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:11.498261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2e731-7c7e-489e-a08f-6cfd5e7d1b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:11.516136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b7f2e731-7c7e-489e-a08f-6cfd5e7d1b9d None None] MFA Rules not processed for user `68f6997db7c141529c37cc64955df779`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:11.517355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b7f2e731-7c7e-489e-a08f-6cfd5e7d1b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:11.576890 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1232/2467] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:14.441636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06135a5c-0760-41f6-aa1f-e5ecbbf9ceee None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:14.443145 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-06135a5c-0760-41f6-aa1f-e5ecbbf9ceee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:14.457407 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06135a5c-0760-41f6-aa1f-e5ecbbf9ceee None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:14.461296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b773fa3-92c0-47f4-9b13-ae44dadb73ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:14.462066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b773fa3-92c0-47f4-9b13-ae44dadb73ac service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:14.462542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b773fa3-92c0-47f4-9b13-ae44dadb73ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:14.462827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b773fa3-92c0-47f4-9b13-ae44dadb73ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:14.469001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0b773fa3-92c0-47f4-9b13-ae44dadb73ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:14.488695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b773fa3-92c0-47f4-9b13-ae44dadb73ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:14.490464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b773fa3-92c0-47f4-9b13-ae44dadb73ac service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:14.521160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1236/2468] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:21.569618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=25 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:24.503448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:28.319102 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2e731-7c7e-489e-a08f-6cfd5e7d1b9d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:28.329462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2e731-7c7e-489e-a08f-6cfd5e7d1b9d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:28.333520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ac037-b45c-4442-9368-ebe2bbeddc18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:28.334218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ac037-b45c-4442-9368-ebe2bbeddc18 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:28.334496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ac037-b45c-4442-9368-ebe2bbeddc18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:28.334756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ac037-b45c-4442-9368-ebe2bbeddc18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:28.336762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dc5ac037-b45c-4442-9368-ebe2bbeddc18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:28.401500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ac037-b45c-4442-9368-ebe2bbeddc18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:28.402668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ac037-b45c-4442-9368-ebe2bbeddc18 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:28.413302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1233/2469] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:32.508012 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b773fa3-92c0-47f4-9b13-ae44dadb73ac service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:32.509750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0b773fa3-92c0-47f4-9b13-ae44dadb73ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:32.526464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b773fa3-92c0-47f4-9b13-ae44dadb73ac service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:32.537591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:32.539914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:32.540449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:32.541113 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] PATH_INFO: `/v3/users/0965444614ae486e80e111f4126b52f1` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:32.559681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0965444614ae486e80e111f4126b52f1)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:32.562121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:32.620428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0965444614ae486e80e111f4126b52f1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:32.627796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0965444614ae486e80e111f4126b52f1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:32.628552 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0965444614ae486e80e111f4126b52f1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:32.639648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0965444614ae486e80e111f4126b52f1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:32.653337 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cae013d1-cd61-4c04-a9fa-3be9c18d3099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0965444614ae486e80e111f4126b52f1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:32.662607 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1237/2470] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:08:32 2022] DELETE /identity/v3/users/0965444614ae486e80e111f4126b52f1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:32.849939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ac037-b45c-4442-9368-ebe2bbeddc18 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:32.862610 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ac037-b45c-4442-9368-ebe2bbeddc18 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:32.925687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:32.926672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:32.927109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:32.927521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] PATH_INFO: `/v3/projects/367a52be93e24b5eaf528a48342d47b3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:32.938434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] RBAC: Authorizing `identity:delete_project(project_id=367a52be93e24b5eaf528a48342d47b3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:32.940418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.001163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367a52be93e24b5eaf528a48342d47b3', 'request_id': 'req-fb3c662b-1c89-4956-8a87-96219d0be01e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:33.009274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367a52be93e24b5eaf528a48342d47b3', 'request_id': 'req-fb3c662b-1c89-4956-8a87-96219d0be01e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:33.020668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367a52be93e24b5eaf528a48342d47b3', 'request_id': 'req-fb3c662b-1c89-4956-8a87-96219d0be01e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:33.033465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] The token cache is being invalidate because project 367a52be93e24b5eaf528a48342d47b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:08:33.033769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:33.034497 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3c662b-1c89-4956-8a87-96219d0be01e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:33.036939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1234/2471] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:08:32 2022] DELETE /identity/v3/projects/367a52be93e24b5eaf528a48342d47b3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:33.043540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14332cc1-e0f8-4935-a715-691c8148f24b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.044202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14332cc1-e0f8-4935-a715-691c8148f24b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.044660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14332cc1-e0f8-4935-a715-691c8148f24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.071969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-14332cc1-e0f8-4935-a715-691c8148f24b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:33.073535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-14332cc1-e0f8-4935-a715-691c8148f24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:33.143816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1238/2472] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:33.145286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e71da-cc4c-4757-b269-299e574e00f2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.145652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e71da-cc4c-4757-b269-299e574e00f2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.145933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e71da-cc4c-4757-b269-299e574e00f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.167159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8f6e71da-cc4c-4757-b269-299e574e00f2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:33.168793 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8f6e71da-cc4c-4757-b269-299e574e00f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:33.181348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1235/2473] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:08:33.182443 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14332cc1-e0f8-4935-a715-691c8148f24b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.192971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14332cc1-e0f8-4935-a715-691c8148f24b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.199836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c0dd4-ee0c-4050-888c-b3199c21fb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.200636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c0dd4-ee0c-4050-888c-b3199c21fb6d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.200926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c0dd4-ee0c-4050-888c-b3199c21fb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.201208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c0dd4-ee0c-4050-888c-b3199c21fb6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.201960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c0dd4-ee0c-4050-888c-b3199c21fb6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:08:33.202272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c0dd4-ee0c-4050-888c-b3199c21fb6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:33.204156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c0dd4-ee0c-4050-888c-b3199c21fb6d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.220161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1239/2474] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 21:08:33.222658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e71da-cc4c-4757-b269-299e574e00f2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.233262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e71da-cc4c-4757-b269-299e574e00f2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.241037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fade2-e312-4081-ad29-356ee7b2ae6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.241707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fade2-e312-4081-ad29-356ee7b2ae6f admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.241987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fade2-e312-4081-ad29-356ee7b2ae6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.242261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fade2-e312-4081-ad29-356ee7b2ae6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.242959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fade2-e312-4081-ad29-356ee7b2ae6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:33.245039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fade2-e312-4081-ad29-356ee7b2ae6f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.254521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:33.286209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1236/2475] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:08:33 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:08:33.290744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c0dd4-ee0c-4050-888c-b3199c21fb6d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.303714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c0dd4-ee0c-4050-888c-b3199c21fb6d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.357248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:33.375595 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e39e5a-a98b-4645-9be7-2e867c5240d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.376395 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e39e5a-a98b-4645-9be7-2e867c5240d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.376809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e39e5a-a98b-4645-9be7-2e867c5240d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.377299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e39e5a-a98b-4645-9be7-2e867c5240d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.378105 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e39e5a-a98b-4645-9be7-2e867c5240d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:33.380593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e39e5a-a98b-4645-9be7-2e867c5240d4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.408665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1240/2476] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:08:33 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:08:33.412428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fade2-e312-4081-ad29-356ee7b2ae6f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.425673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fade2-e312-4081-ad29-356ee7b2ae6f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.435342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14608442-29d7-4ebe-aa6d-7dde3998a405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.437016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14608442-29d7-4ebe-aa6d-7dde3998a405 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.437314 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14608442-29d7-4ebe-aa6d-7dde3998a405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.437623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14608442-29d7-4ebe-aa6d-7dde3998a405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.438569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14608442-29d7-4ebe-aa6d-7dde3998a405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:33.440836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14608442-29d7-4ebe-aa6d-7dde3998a405 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.452740 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1237/2477] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:08:33.456807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e39e5a-a98b-4645-9be7-2e867c5240d4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.467560 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e39e5a-a98b-4645-9be7-2e867c5240d4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.472186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef2e6a-cd1a-4b92-962c-daf7eadcdbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.472855 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef2e6a-cd1a-4b92-962c-daf7eadcdbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.473132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef2e6a-cd1a-4b92-962c-daf7eadcdbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.473403 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef2e6a-cd1a-4b92-962c-daf7eadcdbd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.474180 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef2e6a-cd1a-4b92-962c-daf7eadcdbd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:33.475826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef2e6a-cd1a-4b92-962c-daf7eadcdbd5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.486104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1241/2478] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:08:33.490424 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14608442-29d7-4ebe-aa6d-7dde3998a405 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.500352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14608442-29d7-4ebe-aa6d-7dde3998a405 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.504990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b1e5-688c-4c0d-b16f-d84bbdc7e05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.505905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b1e5-688c-4c0d-b16f-d84bbdc7e05b admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.506180 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b1e5-688c-4c0d-b16f-d84bbdc7e05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.506450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b1e5-688c-4c0d-b16f-d84bbdc7e05b admin admin] PATH_INFO: `/v3/projects/5efb8093c7bc4d54a4760b93b855b605/users/bd6677a390ff49739f235ebde54f75fd/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.521578 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11b1e5-688c-4c0d-b16f-d84bbdc7e05b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5efb8093c7bc4d54a4760b93b855b605, user_id=bd6677a390ff49739f235ebde54f75fd, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:33.524315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11b1e5-688c-4c0d-b16f-d84bbdc7e05b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.539720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1238/2479] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:08:33 2022] PUT /identity/v3/projects/5efb8093c7bc4d54a4760b93b855b605/users/bd6677a390ff49739f235ebde54f75fd/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:08:33.551013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f976747-9a31-4bf8-9999-d9fa639ce60f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.551490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f976747-9a31-4bf8-9999-d9fa639ce60f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.551606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f976747-9a31-4bf8-9999-d9fa639ce60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.582452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5f976747-9a31-4bf8-9999-d9fa639ce60f None None] MFA Rules not processed for user `bd6677a390ff49739f235ebde54f75fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:33.583779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5f976747-9a31-4bf8-9999-d9fa639ce60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:33.665834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1242/2480] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:33.678456 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b1e5-688c-4c0d-b16f-d84bbdc7e05b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.745062 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b1e5-688c-4c0d-b16f-d84bbdc7e05b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.750280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa07559-4ae9-4f47-bf0a-e563e87cc9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.750949 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa07559-4ae9-4f47-bf0a-e563e87cc9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.751226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa07559-4ae9-4f47-bf0a-e563e87cc9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.751499 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa07559-4ae9-4f47-bf0a-e563e87cc9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.752229 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa07559-4ae9-4f47-bf0a-e563e87cc9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:33.754140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa07559-4ae9-4f47-bf0a-e563e87cc9b2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.785431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1239/2481] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:08:33 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:08:33.786586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f976747-9a31-4bf8-9999-d9fa639ce60f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.797632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f976747-9a31-4bf8-9999-d9fa639ce60f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.863780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e059cdb-9da6-4600-81c4-998d468f1fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.864603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e059cdb-9da6-4600-81c4-998d468f1fee admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.864887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e059cdb-9da6-4600-81c4-998d468f1fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.865185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e059cdb-9da6-4600-81c4-998d468f1fee admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.865882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e059cdb-9da6-4600-81c4-998d468f1fee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:33.867846 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e059cdb-9da6-4600-81c4-998d468f1fee admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.888734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1243/2482] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:08:33 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:08:33.892895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa07559-4ae9-4f47-bf0a-e563e87cc9b2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.904908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa07559-4ae9-4f47-bf0a-e563e87cc9b2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.910476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f23cf8-91ec-4b78-b02d-d34f88703994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.911162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f23cf8-91ec-4b78-b02d-d34f88703994 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.911893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f23cf8-91ec-4b78-b02d-d34f88703994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.912213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f23cf8-91ec-4b78-b02d-d34f88703994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.912965 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f23cf8-91ec-4b78-b02d-d34f88703994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:33.914593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f23cf8-91ec-4b78-b02d-d34f88703994 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.925782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1240/2483] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:08:33.929880 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e059cdb-9da6-4600-81c4-998d468f1fee admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.940954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e059cdb-9da6-4600-81c4-998d468f1fee admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.946431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213e7c0-229d-4660-9db6-93347123bfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.947094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213e7c0-229d-4660-9db6-93347123bfcf admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.947385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213e7c0-229d-4660-9db6-93347123bfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.947664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213e7c0-229d-4660-9db6-93347123bfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:33.948422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213e7c0-229d-4660-9db6-93347123bfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:33.950037 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213e7c0-229d-4660-9db6-93347123bfcf admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:33.960416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1244/2484] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:08:33.964967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f23cf8-91ec-4b78-b02d-d34f88703994 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:33.979365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f23cf8-91ec-4b78-b02d-d34f88703994 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:33.985071 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7633ebc-e724-49c5-915d-3016eae80fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:33.986077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7633ebc-e724-49c5-915d-3016eae80fee admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:33.986246 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7633ebc-e724-49c5-915d-3016eae80fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:33.986529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7633ebc-e724-49c5-915d-3016eae80fee admin admin] PATH_INFO: `/v3/projects/60d6d3e42a9e41d593206c0e198a0f6f/users/b8d0986521234f4398b44f4a9239df35/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:34.022595 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7633ebc-e724-49c5-915d-3016eae80fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d6d3e42a9e41d593206c0e198a0f6f, user_id=b8d0986521234f4398b44f4a9239df35, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:34.025563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7633ebc-e724-49c5-915d-3016eae80fee admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:34.044299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1241/2485] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:08:33 2022] PUT /identity/v3/projects/60d6d3e42a9e41d593206c0e198a0f6f/users/b8d0986521234f4398b44f4a9239df35/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:08:34.055897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9ea4f-92fa-4c7a-b2c7-792517b4a702 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:34.056219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9ea4f-92fa-4c7a-b2c7-792517b4a702 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:34.056499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9ea4f-92fa-4c7a-b2c7-792517b4a702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:34.083744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-38a9ea4f-92fa-4c7a-b2c7-792517b4a702 None None] MFA Rules not processed for user `b8d0986521234f4398b44f4a9239df35`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:34.085140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-38a9ea4f-92fa-4c7a-b2c7-792517b4a702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:34.109618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:34.151423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1245/2486] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:34.156289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7633ebc-e724-49c5-915d-3016eae80fee admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:34.218414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7633ebc-e724-49c5-915d-3016eae80fee admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:34.223115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdc33f-5df2-434c-98a6-7e172302db2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:34.223966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdc33f-5df2-434c-98a6-7e172302db2e admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:34.224347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdc33f-5df2-434c-98a6-7e172302db2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:34.224626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdc33f-5df2-434c-98a6-7e172302db2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:34.225333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdc33f-5df2-434c-98a6-7e172302db2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:34.227255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdc33f-5df2-434c-98a6-7e172302db2e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:34.231725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:34.261244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1242/2487] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:08:34 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:08:34.266974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9ea4f-92fa-4c7a-b2c7-792517b4a702 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:34.278399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9ea4f-92fa-4c7a-b2c7-792517b4a702 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:34.337899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd664235-5530-4c63-9adf-3abc3a262b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:34.338448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd664235-5530-4c63-9adf-3abc3a262b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:34.338728 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd664235-5530-4c63-9adf-3abc3a262b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:34.339002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd664235-5530-4c63-9adf-3abc3a262b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:34.339682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd664235-5530-4c63-9adf-3abc3a262b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:34.341656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd664235-5530-4c63-9adf-3abc3a262b2b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:34.362958 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1246/2488] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:08:34 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:08:34.367273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdc33f-5df2-434c-98a6-7e172302db2e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:34.378465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdc33f-5df2-434c-98a6-7e172302db2e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:34.384072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee663128-1fcd-4dda-974a-812242aef2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:34.384776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee663128-1fcd-4dda-974a-812242aef2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:34.385058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee663128-1fcd-4dda-974a-812242aef2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:34.385331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee663128-1fcd-4dda-974a-812242aef2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:34.386092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee663128-1fcd-4dda-974a-812242aef2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:34.387690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee663128-1fcd-4dda-974a-812242aef2a9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:34.398912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1243/2489] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:08:34.404480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd664235-5530-4c63-9adf-3abc3a262b2b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:34.423603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd664235-5530-4c63-9adf-3abc3a262b2b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:34.430458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad672f-e200-48e0-8494-66a3229f435a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:34.431369 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad672f-e200-48e0-8494-66a3229f435a admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:34.431652 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad672f-e200-48e0-8494-66a3229f435a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:34.431914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad672f-e200-48e0-8494-66a3229f435a admin admin] PATH_INFO: `/v3/projects/ca0df979553e4aee96398cf989048f49/users/c3a4282e96a946b5800a097bad17fc87/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:34.448146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad672f-e200-48e0-8494-66a3229f435a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0df979553e4aee96398cf989048f49, user_id=c3a4282e96a946b5800a097bad17fc87, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:34.450993 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad672f-e200-48e0-8494-66a3229f435a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:34.469548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1247/2490] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:08:34 2022] PUT /identity/v3/projects/ca0df979553e4aee96398cf989048f49/users/c3a4282e96a946b5800a097bad17fc87/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:34.471156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee663128-1fcd-4dda-974a-812242aef2a9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:34.482458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee663128-1fcd-4dda-974a-812242aef2a9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:34.559480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3fb8a-44e5-4ad4-b802-ed2a91590f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:34.560206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3fb8a-44e5-4ad4-b802-ed2a91590f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:34.560489 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3fb8a-44e5-4ad4-b802-ed2a91590f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:34.560764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3fb8a-44e5-4ad4-b802-ed2a91590f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:34.561562 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3fb8a-44e5-4ad4-b802-ed2a91590f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:34.563181 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3fb8a-44e5-4ad4-b802-ed2a91590f60 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:34.576147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1244/2491] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:08:34.578488 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad672f-e200-48e0-8494-66a3229f435a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:34.589362 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad672f-e200-48e0-8494-66a3229f435a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:34.594027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ce7d9-a667-46c8-b63f-e371552edf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:34.595439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ce7d9-a667-46c8-b63f-e371552edf29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:34.595439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ce7d9-a667-46c8-b63f-e371552edf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:34.595439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ce7d9-a667-46c8-b63f-e371552edf29 admin admin] PATH_INFO: `/v3/projects/ca0df979553e4aee96398cf989048f49/users/c3a4282e96a946b5800a097bad17fc87/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:34.597547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ce7d9-a667-46c8-b63f-e371552edf29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0df979553e4aee96398cf989048f49, user_id=c3a4282e96a946b5800a097bad17fc87, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:34.600288 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ce7d9-a667-46c8-b63f-e371552edf29 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:34.617126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1248/2492] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:08:34 2022] PUT /identity/v3/projects/ca0df979553e4aee96398cf989048f49/users/c3a4282e96a946b5800a097bad17fc87/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:34.627638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3f6ff-4158-4e5a-af4e-46d79df78dfb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:34.628362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3f6ff-4158-4e5a-af4e-46d79df78dfb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:34.628845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3f6ff-4158-4e5a-af4e-46d79df78dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:34.666726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-15b3f6ff-4158-4e5a-af4e-46d79df78dfb None None] MFA Rules not processed for user `c3a4282e96a946b5800a097bad17fc87`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:34.668064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-15b3f6ff-4158-4e5a-af4e-46d79df78dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:34.741662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1245/2493] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:08:34.752749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ce7d9-a667-46c8-b63f-e371552edf29 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:34.754732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d99ce7d9-a667-46c8-b63f-e371552edf29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:34.820888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ce7d9-a667-46c8-b63f-e371552edf29 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:34.824893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d49dce-811b-4a52-a2d6-075d5ee999ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:34.825557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d49dce-811b-4a52-a2d6-075d5ee999ab service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:34.825832 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d49dce-811b-4a52-a2d6-075d5ee999ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:34.826103 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d49dce-811b-4a52-a2d6-075d5ee999ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:34.837056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d49dce-811b-4a52-a2d6-075d5ee999ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:34.838554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d49dce-811b-4a52-a2d6-075d5ee999ab service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:34.849104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1249/2494] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:35.380142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3f6ff-4158-4e5a-af4e-46d79df78dfb None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:35.381953 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-15b3f6ff-4158-4e5a-af4e-46d79df78dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:35.394278 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3f6ff-4158-4e5a-af4e-46d79df78dfb None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:35.398401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cee254-50db-4525-abb2-df69ad68ff2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:35.399048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cee254-50db-4525-abb2-df69ad68ff2d service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:35.399322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cee254-50db-4525-abb2-df69ad68ff2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:35.399593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cee254-50db-4525-abb2-df69ad68ff2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:35.456366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:35.459398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cee254-50db-4525-abb2-df69ad68ff2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:35.461412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cee254-50db-4525-abb2-df69ad68ff2d service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:35.472267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1246/2495] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:39.531777 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d49dce-811b-4a52-a2d6-075d5ee999ab service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:39.541340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d49dce-811b-4a52-a2d6-075d5ee999ab service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:39.545209 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a6642-1c66-465a-abfd-24d0b8f559bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:39.545879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a6642-1c66-465a-abfd-24d0b8f559bf service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:39.546148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a6642-1c66-465a-abfd-24d0b8f559bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:39.546417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a6642-1c66-465a-abfd-24d0b8f559bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:39.565044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:39.614827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a6642-1c66-465a-abfd-24d0b8f559bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:39.616322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a6642-1c66-465a-abfd-24d0b8f559bf service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:39.625986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1250/2496] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:41.743253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cee254-50db-4525-abb2-df69ad68ff2d service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:41.805931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cee254-50db-4525-abb2-df69ad68ff2d service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:41.811277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:41.812287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:41.812557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:41.812825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] PATH_INFO: `/v3/users/bd6677a390ff49739f235ebde54f75fd` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:41.814122 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd6677a390ff49739f235ebde54f75fd)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:41.819293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:41.868509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd6677a390ff49739f235ebde54f75fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:41.869155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd6677a390ff49739f235ebde54f75fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:41.875976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd6677a390ff49739f235ebde54f75fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:41.876646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd6677a390ff49739f235ebde54f75fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:41.888838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd6677a390ff49739f235ebde54f75fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:41.903556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1247/2497] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:08:41 2022] DELETE /identity/v3/users/bd6677a390ff49739f235ebde54f75fd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:41.909398 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36bd34-086a-4676-8b39-6c1ce8d08228 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:41.909951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36bd34-086a-4676-8b39-6c1ce8d08228 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:41.909951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36bd34-086a-4676-8b39-6c1ce8d08228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:41.927838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cd36bd34-086a-4676-8b39-6c1ce8d08228 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:41.929077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cd36bd34-086a-4676-8b39-6c1ce8d08228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:41.992501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1251/2498] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:08:42.002825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:42.004471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:42.028475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36bd34-086a-4676-8b39-6c1ce8d08228 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:42.030057 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cd36bd34-086a-4676-8b39-6c1ce8d08228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:42.063167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571a0cf-cc01-4609-8ab6-1229b1c2d49d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:42.066830 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f96d60-b5f5-4800-91da-4685b25b3aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:42.067479 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f96d60-b5f5-4800-91da-4685b25b3aca service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:42.067757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f96d60-b5f5-4800-91da-4685b25b3aca service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:42.068028 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f96d60-b5f5-4800-91da-4685b25b3aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:42.077159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f96d60-b5f5-4800-91da-4685b25b3aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:42.078581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f96d60-b5f5-4800-91da-4685b25b3aca service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:42.087924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36bd34-086a-4676-8b39-6c1ce8d08228 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:42.088778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1248/2499] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:42.091836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352598-5358-4404-b6fb-3bdd43d5f434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:42.092521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352598-5358-4404-b6fb-3bdd43d5f434 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:42.092802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352598-5358-4404-b6fb-3bdd43d5f434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:42.093073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352598-5358-4404-b6fb-3bdd43d5f434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:42.094865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-80352598-5358-4404-b6fb-3bdd43d5f434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:42.160825 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80352598-5358-4404-b6fb-3bdd43d5f434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:42.162778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80352598-5358-4404-b6fb-3bdd43d5f434 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:42.173479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1252/2500] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:42.397246 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f96d60-b5f5-4800-91da-4685b25b3aca service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:42.399609 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-71f96d60-b5f5-4800-91da-4685b25b3aca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:42.432503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f96d60-b5f5-4800-91da-4685b25b3aca service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:42.438239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:42.439156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:42.439422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:42.439698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] PATH_INFO: `/v3/projects/5efb8093c7bc4d54a4760b93b855b605` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:42.441029 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5efb8093c7bc4d54a4760b93b855b605)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:42.442993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:42.458816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:42.511383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5efb8093c7bc4d54a4760b93b855b605', 'request_id': 'req-5ae6886e-b960-416d-8932-09005eab875d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:42.519561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5efb8093c7bc4d54a4760b93b855b605', 'request_id': 'req-5ae6886e-b960-416d-8932-09005eab875d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:42.532142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5efb8093c7bc4d54a4760b93b855b605', 'request_id': 'req-5ae6886e-b960-416d-8932-09005eab875d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:42.547941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] The token cache is being invalidate because project 5efb8093c7bc4d54a4760b93b855b605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:08:42.548583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:42.549361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:42.551222 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1249/2501] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:08:42 2022] DELETE /identity/v3/projects/5efb8093c7bc4d54a4760b93b855b605 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:42.555312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352598-5358-4404-b6fb-3bdd43d5f434 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:42.566824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352598-5358-4404-b6fb-3bdd43d5f434 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:42.572475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:42.639540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:42.640471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:42.640754 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:42.641031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] PATH_INFO: `/v3/users/b8d0986521234f4398b44f4a9239df35` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:42.642826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d0986521234f4398b44f4a9239df35)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:42.645264 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:42.701498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d0986521234f4398b44f4a9239df35'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:42.710501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d0986521234f4398b44f4a9239df35'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:42.711385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d0986521234f4398b44f4a9239df35'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:42.720528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d0986521234f4398b44f4a9239df35'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:42.733871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d0986521234f4398b44f4a9239df35'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:42.741475 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1253/2502] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:08:42 2022] DELETE /identity/v3/users/b8d0986521234f4398b44f4a9239df35 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:43.030691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:43.032306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:43.047723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6886e-b960-416d-8932-09005eab875d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:43.110612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:43.111912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:43.112331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:43.112703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] PATH_INFO: `/v3/projects/60d6d3e42a9e41d593206c0e198a0f6f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:43.113991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60d6d3e42a9e41d593206c0e198a0f6f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:43.116661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:43.164635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60d6d3e42a9e41d593206c0e198a0f6f', 'request_id': 'req-dca4e103-3fda-49c6-a159-3ed0e354f0f9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.171325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60d6d3e42a9e41d593206c0e198a0f6f', 'request_id': 'req-dca4e103-3fda-49c6-a159-3ed0e354f0f9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.179781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60d6d3e42a9e41d593206c0e198a0f6f', 'request_id': 'req-dca4e103-3fda-49c6-a159-3ed0e354f0f9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.191402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] The token cache is being invalidate because project 60d6d3e42a9e41d593206c0e198a0f6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:08:43.191682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.192365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.193943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1250/2503] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:08:43 2022] DELETE /identity/v3/projects/60d6d3e42a9e41d593206c0e198a0f6f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:43.198112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:43.199727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:43.212466 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f15735-9f33-4c65-b1cc-10a7ae058203 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:43.270575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:43.271469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:43.271744 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:43.272014 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] PATH_INFO: `/v3/users/c3a4282e96a946b5800a097bad17fc87` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:43.273314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a4282e96a946b5800a097bad17fc87)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:43.275183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:43.312803 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:43.329405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a4282e96a946b5800a097bad17fc87'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.336661 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a4282e96a946b5800a097bad17fc87'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.337382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a4282e96a946b5800a097bad17fc87'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.346116 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a4282e96a946b5800a097bad17fc87'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.359001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6050dc91-7e08-4c2c-803a-d2cceae60ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a4282e96a946b5800a097bad17fc87'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.365195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1254/2504] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:08:43 2022] DELETE /identity/v3/users/c3a4282e96a946b5800a097bad17fc87 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:43.809083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:43.810900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:43.827710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4e103-3fda-49c6-a159-3ed0e354f0f9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:43.868002 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:43.899318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:43.900227 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:43.900506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:43.900781 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] PATH_INFO: `/v3/projects/ca0df979553e4aee96398cf989048f49` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:43.902015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0df979553e4aee96398cf989048f49)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:43.904057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:43.959577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0df979553e4aee96398cf989048f49', 'request_id': 'req-df178da4-20ff-4501-99d2-c9bcc658b9a7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.966871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0df979553e4aee96398cf989048f49', 'request_id': 'req-df178da4-20ff-4501-99d2-c9bcc658b9a7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.976503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0df979553e4aee96398cf989048f49', 'request_id': 'req-df178da4-20ff-4501-99d2-c9bcc658b9a7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.988984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] The token cache is being invalidate because project ca0df979553e4aee96398cf989048f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:08:43.989484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.990209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-df178da4-20ff-4501-99d2-c9bcc658b9a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:43.992318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1251/2505] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:08:43 2022] DELETE /identity/v3/projects/ca0df979553e4aee96398cf989048f49 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:43.998968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e226eb7-f54f-4d8b-8af4-d441211414f9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:43.999267 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e226eb7-f54f-4d8b-8af4-d441211414f9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.000089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e226eb7-f54f-4d8b-8af4-d441211414f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.025514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4e226eb7-f54f-4d8b-8af4-d441211414f9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:44.027109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4e226eb7-f54f-4d8b-8af4-d441211414f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:44.110713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1255/2506] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:44.118908 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8057e7b-3847-4177-b2bd-c62c077235f5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.119406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8057e7b-3847-4177-b2bd-c62c077235f5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.119522 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8057e7b-3847-4177-b2bd-c62c077235f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.139438 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d8057e7b-3847-4177-b2bd-c62c077235f5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:44.140717 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d8057e7b-3847-4177-b2bd-c62c077235f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:44.149313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1252/2507] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:44.154161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e226eb7-f54f-4d8b-8af4-d441211414f9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.164420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e226eb7-f54f-4d8b-8af4-d441211414f9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.169226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb6a1a-863f-492d-910b-58240fabc8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.169924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb6a1a-863f-492d-910b-58240fabc8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.170205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb6a1a-863f-492d-910b-58240fabc8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.170479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb6a1a-863f-492d-910b-58240fabc8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.171232 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb6a1a-863f-492d-910b-58240fabc8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:08:44.171544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb6a1a-863f-492d-910b-58240fabc8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.173605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb6a1a-863f-492d-910b-58240fabc8d1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:44.186689 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1256/2508] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:08:44.191655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8057e7b-3847-4177-b2bd-c62c077235f5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.201943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8057e7b-3847-4177-b2bd-c62c077235f5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.207267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8f136-0aa4-4bdf-9f0d-96e67b7864e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.207836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8f136-0aa4-4bdf-9f0d-96e67b7864e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.208133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8f136-0aa4-4bdf-9f0d-96e67b7864e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.208454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8f136-0aa4-4bdf-9f0d-96e67b7864e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.209146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8f136-0aa4-4bdf-9f0d-96e67b7864e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.211085 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8f136-0aa4-4bdf-9f0d-96e67b7864e6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:44.239620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1253/2509] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:08:44 2022] POST /identity/v3/projects => generated 407 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:08:44.243973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb6a1a-863f-492d-910b-58240fabc8d1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.254542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb6a1a-863f-492d-910b-58240fabc8d1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.316046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84f0b4-a75b-426e-891e-4a5ab2a9de72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.316863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84f0b4-a75b-426e-891e-4a5ab2a9de72 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.317231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84f0b4-a75b-426e-891e-4a5ab2a9de72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.317512 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84f0b4-a75b-426e-891e-4a5ab2a9de72 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.318208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84f0b4-a75b-426e-891e-4a5ab2a9de72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.320832 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84f0b4-a75b-426e-891e-4a5ab2a9de72 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:44.347328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1257/2510] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:08:44 2022] POST /identity/v3/users => generated 304 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:08:44.348700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8f136-0aa4-4bdf-9f0d-96e67b7864e6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.360016 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8f136-0aa4-4bdf-9f0d-96e67b7864e6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.364756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0d0a2-4d89-4f86-a6d9-0dfe507200c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.365435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d0a2-4d89-4f86-a6d9-0dfe507200c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.365712 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d0a2-4d89-4f86-a6d9-0dfe507200c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.365975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d0a2-4d89-4f86-a6d9-0dfe507200c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.366905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0d0a2-4d89-4f86-a6d9-0dfe507200c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.368541 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0d0a2-4d89-4f86-a6d9-0dfe507200c5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:44.380805 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1254/2511] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:08:44.383867 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84f0b4-a75b-426e-891e-4a5ab2a9de72 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.395745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84f0b4-a75b-426e-891e-4a5ab2a9de72 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.400472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf700949-6020-472e-89e9-84f940b59b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.401305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700949-6020-472e-89e9-84f940b59b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.401950 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700949-6020-472e-89e9-84f940b59b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.401950 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700949-6020-472e-89e9-84f940b59b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.402632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf700949-6020-472e-89e9-84f940b59b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.404275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf700949-6020-472e-89e9-84f940b59b8f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:44.415581 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1258/2512] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:08:44.420094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0d0a2-4d89-4f86-a6d9-0dfe507200c5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.430358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0d0a2-4d89-4f86-a6d9-0dfe507200c5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.435084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0a82a-57ca-4a9d-ad4d-2a3c822308eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.435991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0a82a-57ca-4a9d-ad4d-2a3c822308eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.436380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0a82a-57ca-4a9d-ad4d-2a3c822308eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.436656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0a82a-57ca-4a9d-ad4d-2a3c822308eb admin admin] PATH_INFO: `/v3/projects/e2b2943c7d914deabbbf53add2a6c492/users/ec662964c0224126aff487efae206c82/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.453311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0a82a-57ca-4a9d-ad4d-2a3c822308eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b2943c7d914deabbbf53add2a6c492, user_id=ec662964c0224126aff487efae206c82, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.455960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0a82a-57ca-4a9d-ad4d-2a3c822308eb admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:44.470952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1255/2513] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:08:44 2022] PUT /identity/v3/projects/e2b2943c7d914deabbbf53add2a6c492/users/ec662964c0224126aff487efae206c82/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:08:44.484199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411f071-e45a-49bd-a20c-50491a6927c4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.484492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411f071-e45a-49bd-a20c-50491a6927c4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.484774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411f071-e45a-49bd-a20c-50491a6927c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.512410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:44.517249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b411f071-e45a-49bd-a20c-50491a6927c4 None None] MFA Rules not processed for user `ec662964c0224126aff487efae206c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:44.519308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b411f071-e45a-49bd-a20c-50491a6927c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:44.597182 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1259/2514] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:44.601517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0a82a-57ca-4a9d-ad4d-2a3c822308eb admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.668853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:44.673365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0a82a-57ca-4a9d-ad4d-2a3c822308eb admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.679159 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2983531-cf10-4ac7-9550-8c28c5948163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.679831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2983531-cf10-4ac7-9550-8c28c5948163 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.680134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2983531-cf10-4ac7-9550-8c28c5948163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.680443 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2983531-cf10-4ac7-9550-8c28c5948163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.681190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2983531-cf10-4ac7-9550-8c28c5948163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.683819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2983531-cf10-4ac7-9550-8c28c5948163 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:44.712937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1256/2515] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:08:44 2022] POST /identity/v3/projects => generated 409 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:08:44.714931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411f071-e45a-49bd-a20c-50491a6927c4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.726495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411f071-e45a-49bd-a20c-50491a6927c4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.788265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715c778-9399-4e78-b968-bd1b607b65c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.788943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715c778-9399-4e78-b968-bd1b607b65c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.789244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715c778-9399-4e78-b968-bd1b607b65c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.789522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715c778-9399-4e78-b968-bd1b607b65c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.790220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715c778-9399-4e78-b968-bd1b607b65c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.792711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715c778-9399-4e78-b968-bd1b607b65c7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:44.815366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1260/2516] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:08:44 2022] POST /identity/v3/users => generated 311 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:08:44.819395 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2983531-cf10-4ac7-9550-8c28c5948163 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.836565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2983531-cf10-4ac7-9550-8c28c5948163 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.842332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773cafa-7b70-4aa6-8be0-4fba5ae208d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.843021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773cafa-7b70-4aa6-8be0-4fba5ae208d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.843305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773cafa-7b70-4aa6-8be0-4fba5ae208d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.843580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773cafa-7b70-4aa6-8be0-4fba5ae208d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.844352 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773cafa-7b70-4aa6-8be0-4fba5ae208d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.846015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773cafa-7b70-4aa6-8be0-4fba5ae208d7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:44.858178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1257/2517] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:08:44.862515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715c778-9399-4e78-b968-bd1b607b65c7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.873331 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715c778-9399-4e78-b968-bd1b607b65c7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.878015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807b28f-9b85-4537-8a4b-7c3f7180609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.878951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807b28f-9b85-4537-8a4b-7c3f7180609e admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.879233 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807b28f-9b85-4537-8a4b-7c3f7180609e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.879511 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807b28f-9b85-4537-8a4b-7c3f7180609e admin admin] PATH_INFO: `/v3/projects/24c2e529a9664e29b73ec7ba291abdd9/users/8570a96fc1734982b89d5cd2e33f2c3a/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.894830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807b28f-9b85-4537-8a4b-7c3f7180609e admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c2e529a9664e29b73ec7ba291abdd9, user_id=8570a96fc1734982b89d5cd2e33f2c3a, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.897586 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807b28f-9b85-4537-8a4b-7c3f7180609e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:44.911839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1261/2518] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:08:44 2022] PUT /identity/v3/projects/24c2e529a9664e29b73ec7ba291abdd9/users/8570a96fc1734982b89d5cd2e33f2c3a/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:08:44.916253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773cafa-7b70-4aa6-8be0-4fba5ae208d7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:44.932613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773cafa-7b70-4aa6-8be0-4fba5ae208d7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:44.995657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41748d7-d515-463c-a1dc-429ebe13d75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:44.996021 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41748d7-d515-463c-a1dc-429ebe13d75e admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:44.996327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41748d7-d515-463c-a1dc-429ebe13d75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:44.996677 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41748d7-d515-463c-a1dc-429ebe13d75e admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:44.997428 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41748d7-d515-463c-a1dc-429ebe13d75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:44.999130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41748d7-d515-463c-a1dc-429ebe13d75e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:45.010200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1258/2519] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:08:45.014441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807b28f-9b85-4537-8a4b-7c3f7180609e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:45.025222 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807b28f-9b85-4537-8a4b-7c3f7180609e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:45.030596 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3e3e1-6d6e-4db2-9681-c15e433fd765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:45.031500 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3e3e1-6d6e-4db2-9681-c15e433fd765 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:45.031783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3e3e1-6d6e-4db2-9681-c15e433fd765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:45.032062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3e3e1-6d6e-4db2-9681-c15e433fd765 admin admin] PATH_INFO: `/v3/projects/24c2e529a9664e29b73ec7ba291abdd9/users/8570a96fc1734982b89d5cd2e33f2c3a/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:45.034078 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3e3e1-6d6e-4db2-9681-c15e433fd765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c2e529a9664e29b73ec7ba291abdd9, user_id=8570a96fc1734982b89d5cd2e33f2c3a, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:45.036851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3e3e1-6d6e-4db2-9681-c15e433fd765 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:45.049761 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1262/2520] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:08:45 2022] PUT /identity/v3/projects/24c2e529a9664e29b73ec7ba291abdd9/users/8570a96fc1734982b89d5cd2e33f2c3a/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:45.060160 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bae96-3ff2-4480-9043-4905489b80b8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:45.060457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bae96-3ff2-4480-9043-4905489b80b8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:45.060739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bae96-3ff2-4480-9043-4905489b80b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:45.087754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0d4bae96-3ff2-4480-9043-4905489b80b8 None None] MFA Rules not processed for user `8570a96fc1734982b89d5cd2e33f2c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:45.089197 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0d4bae96-3ff2-4480-9043-4905489b80b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:45.161492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1259/2521] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:45 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:45.173436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3e3e1-6d6e-4db2-9681-c15e433fd765 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:45.175374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-98d3e3e1-6d6e-4db2-9681-c15e433fd765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:45.237542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3e3e1-6d6e-4db2-9681-c15e433fd765 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:45.241786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4eecd-5b14-4018-8b59-52dae268c214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:45.242441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4eecd-5b14-4018-8b59-52dae268c214 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:45.242730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4eecd-5b14-4018-8b59-52dae268c214 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:45.242971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4eecd-5b14-4018-8b59-52dae268c214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:45.252585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4eecd-5b14-4018-8b59-52dae268c214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:45.254030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4eecd-5b14-4018-8b59-52dae268c214 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:45.258052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:45.265627 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1263/2522] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:45 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:45.427936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bae96-3ff2-4480-9043-4905489b80b8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:45.438779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bae96-3ff2-4480-9043-4905489b80b8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:45.443030 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771c3bc-2dcc-4c4c-b86b-b869216fc8c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:45.443673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771c3bc-2dcc-4c4c-b86b-b869216fc8c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:45.443984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771c3bc-2dcc-4c4c-b86b-b869216fc8c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:45.444283 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771c3bc-2dcc-4c4c-b86b-b869216fc8c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:45.494211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:45.505748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771c3bc-2dcc-4c4c-b86b-b869216fc8c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:45.507194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771c3bc-2dcc-4c4c-b86b-b869216fc8c9 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:45.516481 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1260/2523] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:45 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:54.495984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4eecd-5b14-4018-8b59-52dae268c214 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:54.498290 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-43a4eecd-5b14-4018-8b59-52dae268c214 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:54.510952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4eecd-5b14-4018-8b59-52dae268c214 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:54.515591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de64ad2-f929-4988-80f8-cb18dc2f5420 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:54.516275 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de64ad2-f929-4988-80f8-cb18dc2f5420 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:54.516547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de64ad2-f929-4988-80f8-cb18dc2f5420 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:54.516822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de64ad2-f929-4988-80f8-cb18dc2f5420 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:54.518820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7de64ad2-f929-4988-80f8-cb18dc2f5420 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:54.615354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de64ad2-f929-4988-80f8-cb18dc2f5420 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:54.616847 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de64ad2-f929-4988-80f8-cb18dc2f5420 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:54.637026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1264/2524] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:55.512635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:55.998966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771c3bc-2dcc-4c4c-b86b-b869216fc8c9 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:56.066593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771c3bc-2dcc-4c4c-b86b-b869216fc8c9 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:56.071636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:56.072539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:56.072828 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:56.073097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] PATH_INFO: `/v3/users/ec662964c0224126aff487efae206c82` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:56.074394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec662964c0224126aff487efae206c82)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:56.076379 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:56.129211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec662964c0224126aff487efae206c82'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.129898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec662964c0224126aff487efae206c82'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.137251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec662964c0224126aff487efae206c82'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.137902 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec662964c0224126aff487efae206c82'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.150380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec662964c0224126aff487efae206c82'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.168529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1261/2525] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:08:55 2022] DELETE /identity/v3/users/ec662964c0224126aff487efae206c82 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:56.173779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81817151-b464-4e9d-af49-7a8018a317b0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:56.174283 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81817151-b464-4e9d-af49-7a8018a317b0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:56.174412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81817151-b464-4e9d-af49-7a8018a317b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:56.191986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-81817151-b464-4e9d-af49-7a8018a317b0 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:56.193254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-81817151-b464-4e9d-af49-7a8018a317b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:56.260405 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1265/2526] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:56.270347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:56.271801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:56.344509 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698ad07-9441-42d5-bf0c-8cd78a8cf8b7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:56.348936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e000f7-2ab9-4dcd-8d60-c3d6ae63fc8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:56.349588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e000f7-2ab9-4dcd-8d60-c3d6ae63fc8e service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:56.349870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e000f7-2ab9-4dcd-8d60-c3d6ae63fc8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:56.350136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e000f7-2ab9-4dcd-8d60-c3d6ae63fc8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:56.359579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e000f7-2ab9-4dcd-8d60-c3d6ae63fc8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:56.361140 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e000f7-2ab9-4dcd-8d60-c3d6ae63fc8e service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:56.370933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1262/2527] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:56.651008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81817151-b464-4e9d-af49-7a8018a317b0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:56.652581 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-81817151-b464-4e9d-af49-7a8018a317b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:56.665901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81817151-b464-4e9d-af49-7a8018a317b0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:56.674146 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:56.675079 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:56.675366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:56.675643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] PATH_INFO: `/v3/projects/e2b2943c7d914deabbbf53add2a6c492` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:56.676950 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b2943c7d914deabbbf53add2a6c492)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:56.679093 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:56.726520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b2943c7d914deabbbf53add2a6c492', 'request_id': 'req-b64027a9-a258-4ccd-b380-dcd99825f4ab'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.734968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b2943c7d914deabbbf53add2a6c492', 'request_id': 'req-b64027a9-a258-4ccd-b380-dcd99825f4ab'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.743197 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b2943c7d914deabbbf53add2a6c492', 'request_id': 'req-b64027a9-a258-4ccd-b380-dcd99825f4ab'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.745249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:56.756226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] The token cache is being invalidate because project e2b2943c7d914deabbbf53add2a6c492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:08:56.756568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.757136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.758697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1266/2528] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:08:56 2022] DELETE /identity/v3/projects/e2b2943c7d914deabbbf53add2a6c492 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:56.762751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e000f7-2ab9-4dcd-8d60-c3d6ae63fc8e service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:56.773051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e000f7-2ab9-4dcd-8d60-c3d6ae63fc8e service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:56.792307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:56.836050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:56.837060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:56.837330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:56.837594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] PATH_INFO: `/v3/users/8570a96fc1734982b89d5cd2e33f2c3a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:56.838858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8570a96fc1734982b89d5cd2e33f2c3a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:56.840867 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:56.892851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8570a96fc1734982b89d5cd2e33f2c3a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.893510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8570a96fc1734982b89d5cd2e33f2c3a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.900833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8570a96fc1734982b89d5cd2e33f2c3a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.901417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8570a96fc1734982b89d5cd2e33f2c3a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.913502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3734d5dd-5097-4bdd-999e-154ed7bc13ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8570a96fc1734982b89d5cd2e33f2c3a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:56.927482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1263/2529] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:08:56 2022] DELETE /identity/v3/users/8570a96fc1734982b89d5cd2e33f2c3a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:57.208580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:57.210045 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:57.224853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64027a9-a258-4ccd-b380-dcd99825f4ab admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:57.290388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:57.291274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:57.291553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:57.291827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] PATH_INFO: `/v3/projects/24c2e529a9664e29b73ec7ba291abdd9` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:57.293108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c2e529a9664e29b73ec7ba291abdd9)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:57.295082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:57.355775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c2e529a9664e29b73ec7ba291abdd9', 'request_id': 'req-4d4e44f9-6591-49c5-bddb-47d5d463a98b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:57.365052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c2e529a9664e29b73ec7ba291abdd9', 'request_id': 'req-4d4e44f9-6591-49c5-bddb-47d5d463a98b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:57.373929 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c2e529a9664e29b73ec7ba291abdd9', 'request_id': 'req-4d4e44f9-6591-49c5-bddb-47d5d463a98b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:57.388522 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] The token cache is being invalidate because project 24c2e529a9664e29b73ec7ba291abdd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:08:57.388928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:57.389657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d4e44f9-6591-49c5-bddb-47d5d463a98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:08:57.391571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1267/2530] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:08:57 2022] DELETE /identity/v3/projects/24c2e529a9664e29b73ec7ba291abdd9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:57.397435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5cad8-8c77-4944-b841-18abf3c7998a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:57.397730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5cad8-8c77-4944-b841-18abf3c7998a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:57.398006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5cad8-8c77-4944-b841-18abf3c7998a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:57.416480 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-49b5cad8-8c77-4944-b841-18abf3c7998a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:57.417812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-49b5cad8-8c77-4944-b841-18abf3c7998a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:57.489148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1264/2531] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:57.498124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46ad49-edd3-4f7e-9e6f-9b46b9353123 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:57.498409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46ad49-edd3-4f7e-9e6f-9b46b9353123 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:57.498683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46ad49-edd3-4f7e-9e6f-9b46b9353123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:57.518052 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3c46ad49-edd3-4f7e-9e6f-9b46b9353123 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:57.519313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3c46ad49-edd3-4f7e-9e6f-9b46b9353123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:57.527323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1268/2532] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:57.532132 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5cad8-8c77-4944-b841-18abf3c7998a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:57.543251 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5cad8-8c77-4944-b841-18abf3c7998a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:57.549165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afd077-2549-4955-9e5a-b8c6edec8e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:57.549849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afd077-2549-4955-9e5a-b8c6edec8e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:57.550124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afd077-2549-4955-9e5a-b8c6edec8e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:57.550388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afd077-2549-4955-9e5a-b8c6edec8e43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:57.551131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afd077-2549-4955-9e5a-b8c6edec8e43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:08:57.551459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afd077-2549-4955-9e5a-b8c6edec8e43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:57.553124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afd077-2549-4955-9e5a-b8c6edec8e43 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:57.567762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1265/2533] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:08:57.572457 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46ad49-edd3-4f7e-9e6f-9b46b9353123 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:57.582603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46ad49-edd3-4f7e-9e6f-9b46b9353123 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:57.587546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058aaac-910f-49f4-b9a0-29e507f9b197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:57.588251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058aaac-910f-49f4-b9a0-29e507f9b197 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:57.588540 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058aaac-910f-49f4-b9a0-29e507f9b197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:57.588860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058aaac-910f-49f4-b9a0-29e507f9b197 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:57.589551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058aaac-910f-49f4-b9a0-29e507f9b197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:57.591462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058aaac-910f-49f4-b9a0-29e507f9b197 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:57.618827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1269/2534] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:08:57 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:08:57.623253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afd077-2549-4955-9e5a-b8c6edec8e43 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:57.633886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afd077-2549-4955-9e5a-b8c6edec8e43 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:57.666652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:57.702912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba8e1d-2e53-41d7-ab6f-87e4e0910f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:57.703658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8e1d-2e53-41d7-ab6f-87e4e0910f94 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:57.703893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8e1d-2e53-41d7-ab6f-87e4e0910f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:57.704225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8e1d-2e53-41d7-ab6f-87e4e0910f94 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:57.705049 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba8e1d-2e53-41d7-ab6f-87e4e0910f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:57.706998 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba8e1d-2e53-41d7-ab6f-87e4e0910f94 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:57.733560 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1266/2535] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:08:57 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:08:57.738087 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058aaac-910f-49f4-b9a0-29e507f9b197 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:57.748814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058aaac-910f-49f4-b9a0-29e507f9b197 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:57.754038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58178e9b-48c2-4aca-98bc-9bbe2ab86c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:57.754717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58178e9b-48c2-4aca-98bc-9bbe2ab86c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:57.755000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58178e9b-48c2-4aca-98bc-9bbe2ab86c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:57.755276 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58178e9b-48c2-4aca-98bc-9bbe2ab86c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:57.756009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58178e9b-48c2-4aca-98bc-9bbe2ab86c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:57.757690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58178e9b-48c2-4aca-98bc-9bbe2ab86c59 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:57.768647 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1270/2536] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:08:57.772772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba8e1d-2e53-41d7-ab6f-87e4e0910f94 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:57.783109 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba8e1d-2e53-41d7-ab6f-87e4e0910f94 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:57.787663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359008f4-1e0d-497a-acc1-45894674a3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:57.788354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359008f4-1e0d-497a-acc1-45894674a3bf admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:57.788667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359008f4-1e0d-497a-acc1-45894674a3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:57.788895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359008f4-1e0d-497a-acc1-45894674a3bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:57.789631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359008f4-1e0d-497a-acc1-45894674a3bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:57.791274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359008f4-1e0d-497a-acc1-45894674a3bf admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:57.805046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1267/2537] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:08:57.806259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58178e9b-48c2-4aca-98bc-9bbe2ab86c59 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:57.816789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58178e9b-48c2-4aca-98bc-9bbe2ab86c59 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:57.821612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a07f8-1184-4ac2-a7c0-05b88d5d7a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:57.822637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a07f8-1184-4ac2-a7c0-05b88d5d7a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:57.822999 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a07f8-1184-4ac2-a7c0-05b88d5d7a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:57.823483 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a07f8-1184-4ac2-a7c0-05b88d5d7a10 admin admin] PATH_INFO: `/v3/projects/f9b22c49f6894d19b15fb2207da10a58/users/b6394259eb2d4b3fb19c9ca6c9741d10/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:57.844546 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a07f8-1184-4ac2-a7c0-05b88d5d7a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b22c49f6894d19b15fb2207da10a58, user_id=b6394259eb2d4b3fb19c9ca6c9741d10, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:57.847339 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a07f8-1184-4ac2-a7c0-05b88d5d7a10 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:57.861399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1271/2538] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:08:57 2022] PUT /identity/v3/projects/f9b22c49f6894d19b15fb2207da10a58/users/b6394259eb2d4b3fb19c9ca6c9741d10/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:08:57.871798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb60ae8-3721-4456-95ab-3cecdd53e4a7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:57.872148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb60ae8-3721-4456-95ab-3cecdd53e4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:57.872419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb60ae8-3721-4456-95ab-3cecdd53e4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:57.900948 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0bb60ae8-3721-4456-95ab-3cecdd53e4a7 None None] MFA Rules not processed for user `b6394259eb2d4b3fb19c9ca6c9741d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:08:57.902054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0bb60ae8-3721-4456-95ab-3cecdd53e4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:57.968670 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a07f8-1184-4ac2-a7c0-05b88d5d7a10 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:57.970649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0a5a07f8-1184-4ac2-a7c0-05b88d5d7a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:57.976088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1268/2539] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:08:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:08:57.988722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb60ae8-3721-4456-95ab-3cecdd53e4a7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:08:57.990229 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0bb60ae8-3721-4456-95ab-3cecdd53e4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:57.994447 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:08:58.062071 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a07f8-1184-4ac2-a7c0-05b88d5d7a10 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:58.078149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acbd3a-756e-43df-aabc-aa00b8dd3fe9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:58.078840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acbd3a-756e-43df-aabc-aa00b8dd3fe9 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:58.079122 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acbd3a-756e-43df-aabc-aa00b8dd3fe9 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:58.079451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acbd3a-756e-43df-aabc-aa00b8dd3fe9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:58.082871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c2acbd3a-756e-43df-aabc-aa00b8dd3fe9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:08:58.102691 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2acbd3a-756e-43df-aabc-aa00b8dd3fe9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:58.104639 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2acbd3a-756e-43df-aabc-aa00b8dd3fe9 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:58.108612 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb60ae8-3721-4456-95ab-3cecdd53e4a7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:08:58.116822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7caf42-a005-459a-8d2c-72bf85eb8aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:08:58.117545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7caf42-a005-459a-8d2c-72bf85eb8aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:08:58.117825 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7caf42-a005-459a-8d2c-72bf85eb8aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:08:58.118287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1272/2540] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:08:58.118415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7caf42-a005-459a-8d2c-72bf85eb8aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:08:58.128545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7caf42-a005-459a-8d2c-72bf85eb8aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:08:58.129996 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7caf42-a005-459a-8d2c-72bf85eb8aa8 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:08:58.142233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1269/2541] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:08:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 21:09:01.214239 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acbd3a-756e-43df-aabc-aa00b8dd3fe9 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:01.283019 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acbd3a-756e-43df-aabc-aa00b8dd3fe9 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:01.288185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:01.289074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:01.289354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:01.289630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] PATH_INFO: `/v3/users/b6394259eb2d4b3fb19c9ca6c9741d10` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:01.290842 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6394259eb2d4b3fb19c9ca6c9741d10)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:01.292818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:01.350437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6394259eb2d4b3fb19c9ca6c9741d10'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:01.358152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6394259eb2d4b3fb19c9ca6c9741d10'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:01.358839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6394259eb2d4b3fb19c9ca6c9741d10'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:01.368671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6394259eb2d4b3fb19c9ca6c9741d10'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:01.384537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6394259eb2d4b3fb19c9ca6c9741d10'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:01.391365 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1273/2542] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:09:01 2022] DELETE /identity/v3/users/b6394259eb2d4b3fb19c9ca6c9741d10 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:01.396309 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda6ab5-6ed3-4826-9e4b-8c9a94b6af40 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:01.396845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda6ab5-6ed3-4826-9e4b-8c9a94b6af40 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:01.397147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda6ab5-6ed3-4826-9e4b-8c9a94b6af40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:01.415230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-dfda6ab5-6ed3-4826-9e4b-8c9a94b6af40 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:01.416713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dfda6ab5-6ed3-4826-9e4b-8c9a94b6af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:01.452879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:01.480738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1270/2543] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:09:01.490956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:01.492698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:01.523555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:01.562559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d211c1-a09c-4e5e-9d24-94bcd4738145 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:01.566367 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f047025-bb31-4c63-9b6b-dfe5f4410ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:01.567005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f047025-bb31-4c63-9b6b-dfe5f4410ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:01.567299 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f047025-bb31-4c63-9b6b-dfe5f4410ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:01.567572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f047025-bb31-4c63-9b6b-dfe5f4410ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:01.577112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f047025-bb31-4c63-9b6b-dfe5f4410ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:01.578555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f047025-bb31-4c63-9b6b-dfe5f4410ee5 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:01.588437 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1274/2544] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:09:01.874594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda6ab5-6ed3-4826-9e4b-8c9a94b6af40 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:01.877136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dfda6ab5-6ed3-4826-9e4b-8c9a94b6af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:01.896460 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda6ab5-6ed3-4826-9e4b-8c9a94b6af40 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:01.902207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:01.903171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:01.903356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:01.903616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] PATH_INFO: `/v3/projects/f9b22c49f6894d19b15fb2207da10a58` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:01.905041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9b22c49f6894d19b15fb2207da10a58)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:01.906864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:01.965144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9b22c49f6894d19b15fb2207da10a58', 'request_id': 'req-355df41a-42f2-4f38-a024-b6400d2c32c9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:01.971840 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9b22c49f6894d19b15fb2207da10a58', 'request_id': 'req-355df41a-42f2-4f38-a024-b6400d2c32c9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:01.980491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9b22c49f6894d19b15fb2207da10a58', 'request_id': 'req-355df41a-42f2-4f38-a024-b6400d2c32c9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:01.990952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] The token cache is being invalidate because project f9b22c49f6894d19b15fb2207da10a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:09:01.991238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:01.991845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-355df41a-42f2-4f38-a024-b6400d2c32c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:01.993476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1271/2545] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:09:01 2022] DELETE /identity/v3/projects/f9b22c49f6894d19b15fb2207da10a58 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:01.998412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f3979-2aa8-411b-8479-a75a295e6164 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:01.998697 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f3979-2aa8-411b-8479-a75a295e6164 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:01.998970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f3979-2aa8-411b-8479-a75a295e6164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.017427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1f8f3979-2aa8-411b-8479-a75a295e6164 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:02.018755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1f8f3979-2aa8-411b-8479-a75a295e6164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:02.082630 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1275/2546] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:02.090669 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e22fd-21d6-4b70-99a6-9d5decde75a2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:02.090943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e22fd-21d6-4b70-99a6-9d5decde75a2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:02.091423 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e22fd-21d6-4b70-99a6-9d5decde75a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.111836 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e39e22fd-21d6-4b70-99a6-9d5decde75a2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:02.113174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e39e22fd-21d6-4b70-99a6-9d5decde75a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:02.123313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1272/2547] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:09:02.128055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f3979-2aa8-411b-8479-a75a295e6164 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:02.138860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f3979-2aa8-411b-8479-a75a295e6164 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:02.145435 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac14f6d-3664-473b-8cd8-f0206f5c7b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:02.146112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac14f6d-3664-473b-8cd8-f0206f5c7b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:02.146377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac14f6d-3664-473b-8cd8-f0206f5c7b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:02.146653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac14f6d-3664-473b-8cd8-f0206f5c7b73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.147393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac14f6d-3664-473b-8cd8-f0206f5c7b73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:09:02.147701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac14f6d-3664-473b-8cd8-f0206f5c7b73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:02.149480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac14f6d-3664-473b-8cd8-f0206f5c7b73 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:02.163791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1276/2548] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:09:02.168782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e22fd-21d6-4b70-99a6-9d5decde75a2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:02.179431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e22fd-21d6-4b70-99a6-9d5decde75a2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:02.185090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d094ed-b5a1-4316-b8ee-b465f629039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:02.185743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d094ed-b5a1-4316-b8ee-b465f629039d admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:02.186025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d094ed-b5a1-4316-b8ee-b465f629039d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:02.186348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d094ed-b5a1-4316-b8ee-b465f629039d admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.187020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d094ed-b5a1-4316-b8ee-b465f629039d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:02.188967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d094ed-b5a1-4316-b8ee-b465f629039d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:02.216772 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1273/2549] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:09:02 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:02.220954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac14f6d-3664-473b-8cd8-f0206f5c7b73 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:02.232418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac14f6d-3664-473b-8cd8-f0206f5c7b73 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:02.294204 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd5f76-c711-4e9e-80aa-32ed4acd5c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:02.294884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd5f76-c711-4e9e-80aa-32ed4acd5c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:02.295169 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd5f76-c711-4e9e-80aa-32ed4acd5c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:02.295446 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd5f76-c711-4e9e-80aa-32ed4acd5c75 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.296141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd5f76-c711-4e9e-80aa-32ed4acd5c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:02.298468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd5f76-c711-4e9e-80aa-32ed4acd5c75 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:02.322456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1277/2550] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:09:02 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:02.326780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d094ed-b5a1-4316-b8ee-b465f629039d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:02.337841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d094ed-b5a1-4316-b8ee-b465f629039d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:02.343345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b33bf-db42-4815-942b-bc5f7021a2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:02.344007 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b33bf-db42-4815-942b-bc5f7021a2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:02.344337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b33bf-db42-4815-942b-bc5f7021a2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:02.344618 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b33bf-db42-4815-942b-bc5f7021a2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.345350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b33bf-db42-4815-942b-bc5f7021a2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:02.347050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b33bf-db42-4815-942b-bc5f7021a2a6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:02.357887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1274/2551] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:02.362438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd5f76-c711-4e9e-80aa-32ed4acd5c75 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:02.373988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd5f76-c711-4e9e-80aa-32ed4acd5c75 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:02.381330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9c264-c93a-4bdb-b471-1c95fc17e751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:02.382295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9c264-c93a-4bdb-b471-1c95fc17e751 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:02.382671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9c264-c93a-4bdb-b471-1c95fc17e751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:02.383033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9c264-c93a-4bdb-b471-1c95fc17e751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.384174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9c264-c93a-4bdb-b471-1c95fc17e751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:02.386711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9c264-c93a-4bdb-b471-1c95fc17e751 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:02.399718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1278/2552] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:02.404053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b33bf-db42-4815-942b-bc5f7021a2a6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:02.414719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b33bf-db42-4815-942b-bc5f7021a2a6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:02.419802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6691f5-c002-4204-a438-78c8a324ad28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:02.420715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6691f5-c002-4204-a438-78c8a324ad28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:02.421000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6691f5-c002-4204-a438-78c8a324ad28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:02.421271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6691f5-c002-4204-a438-78c8a324ad28 admin admin] PATH_INFO: `/v3/projects/3539cabdbfa943a49368f866270a776b/users/fe0be3efded64c04ad05b984b52c534f/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.437921 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6691f5-c002-4204-a438-78c8a324ad28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3539cabdbfa943a49368f866270a776b, user_id=fe0be3efded64c04ad05b984b52c534f, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:02.440661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6691f5-c002-4204-a438-78c8a324ad28 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:02.458237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1275/2553] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:09:02 2022] PUT /identity/v3/projects/3539cabdbfa943a49368f866270a776b/users/fe0be3efded64c04ad05b984b52c534f/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:02.469711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a96c75-e264-441f-a607-5eb152d2c073 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:02.469939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a96c75-e264-441f-a607-5eb152d2c073 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:02.470217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a96c75-e264-441f-a607-5eb152d2c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.499456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d0a96c75-e264-441f-a607-5eb152d2c073 None None] MFA Rules not processed for user `fe0be3efded64c04ad05b984b52c534f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:02.500694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d0a96c75-e264-441f-a607-5eb152d2c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:02.520577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:02.570911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1279/2554] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:02.582294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6691f5-c002-4204-a438-78c8a324ad28 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:02.585008 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-da6691f5-c002-4204-a438-78c8a324ad28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:02.647567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6691f5-c002-4204-a438-78c8a324ad28 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:02.651524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3b5b0-171d-4cc3-ac69-aa45dc678455 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:02.652200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3b5b0-171d-4cc3-ac69-aa45dc678455 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:02.652476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3b5b0-171d-4cc3-ac69-aa45dc678455 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:02.652745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3b5b0-171d-4cc3-ac69-aa45dc678455 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.661378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3b5b0-171d-4cc3-ac69-aa45dc678455 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:02.662812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3b5b0-171d-4cc3-ac69-aa45dc678455 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:02.673069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1276/2555] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:09:02.805068 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a96c75-e264-441f-a607-5eb152d2c073 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:02.806764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d0a96c75-e264-441f-a607-5eb152d2c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:02.873391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a96c75-e264-441f-a607-5eb152d2c073 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:02.876972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee7918-2d17-4d00-842b-c735949b04ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:02.877622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee7918-2d17-4d00-842b-c735949b04ec service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:02.877898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee7918-2d17-4d00-842b-c735949b04ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:02.878163 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee7918-2d17-4d00-842b-c735949b04ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:02.880406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9bee7918-2d17-4d00-842b-c735949b04ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:02.972142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee7918-2d17-4d00-842b-c735949b04ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:02.974112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee7918-2d17-4d00-842b-c735949b04ec service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:02.986790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1280/2556] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:09:04.057308 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd4cec-568f-43eb-b28d-2077c0537858 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:04.057767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd4cec-568f-43eb-b28d-2077c0537858 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:04.057913 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd4cec-568f-43eb-b28d-2077c0537858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:04.064048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:04.077822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e6bd4cec-568f-43eb-b28d-2077c0537858 None None] MFA Rules not processed for user `fe0be3efded64c04ad05b984b52c534f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:04.079119 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e6bd4cec-568f-43eb-b28d-2077c0537858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:04.086246 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1277/2557] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:04.981148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee7918-2d17-4d00-842b-c735949b04ec service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:04.982790 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9bee7918-2d17-4d00-842b-c735949b04ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:05.071195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee7918-2d17-4d00-842b-c735949b04ec service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:05.076715 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272a258-4003-42d5-a7e1-d87ca7e835dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:05.077409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a258-4003-42d5-a7e1-d87ca7e835dd service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:05.077690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a258-4003-42d5-a7e1-d87ca7e835dd service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:05.077964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a258-4003-42d5-a7e1-d87ca7e835dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:05.079768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c272a258-4003-42d5-a7e1-d87ca7e835dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:05.092321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272a258-4003-42d5-a7e1-d87ca7e835dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:05.093963 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272a258-4003-42d5-a7e1-d87ca7e835dd service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:05.101399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:05.107205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1281/2558] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 21:09:06.838539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd4cec-568f-43eb-b28d-2077c0537858 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:06.840043 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e6bd4cec-568f-43eb-b28d-2077c0537858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:06.856960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd4cec-568f-43eb-b28d-2077c0537858 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:06.862255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:06.863277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:06.863558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:06.863931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] PATH_INFO: `/v3/users/77cf43c378a9413dbd799a00bfa21eb8` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:06.865174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77cf43c378a9413dbd799a00bfa21eb8)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:06.867185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:06.918571 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77cf43c378a9413dbd799a00bfa21eb8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:06.919361 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77cf43c378a9413dbd799a00bfa21eb8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:06.927470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77cf43c378a9413dbd799a00bfa21eb8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:06.928228 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77cf43c378a9413dbd799a00bfa21eb8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:06.943096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-74de463a-ac78-44bb-b940-860e0d2a40e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77cf43c378a9413dbd799a00bfa21eb8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:06.956208 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1278/2559] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:09:06 2022] DELETE /identity/v3/users/77cf43c378a9413dbd799a00bfa21eb8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:07.133409 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272a258-4003-42d5-a7e1-d87ca7e835dd service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:07.145058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272a258-4003-42d5-a7e1-d87ca7e835dd service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:07.203676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:07.204600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.204884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.205159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] PATH_INFO: `/v3/projects/570d2b4d5c384612ba17efc8b5bd02ec` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:07.206352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=570d2b4d5c384612ba17efc8b5bd02ec)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:07.208333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:07.260411 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570d2b4d5c384612ba17efc8b5bd02ec', 'request_id': 'req-96f98407-e0fe-4014-bdb5-720c4fc83130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:07.267107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570d2b4d5c384612ba17efc8b5bd02ec', 'request_id': 'req-96f98407-e0fe-4014-bdb5-720c4fc83130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:07.276806 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570d2b4d5c384612ba17efc8b5bd02ec', 'request_id': 'req-96f98407-e0fe-4014-bdb5-720c4fc83130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:07.290245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] The token cache is being invalidate because project 570d2b4d5c384612ba17efc8b5bd02ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:09:07.290618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:07.291183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-96f98407-e0fe-4014-bdb5-720c4fc83130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:07.293152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1282/2560] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:09:07 2022] DELETE /identity/v3/projects/570d2b4d5c384612ba17efc8b5bd02ec => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:07.298272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b1b73-7059-4388-a7dd-c71ceae401d5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.298633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b1b73-7059-4388-a7dd-c71ceae401d5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.298974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b1b73-7059-4388-a7dd-c71ceae401d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:07.324506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-066b1b73-7059-4388-a7dd-c71ceae401d5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:07.325928 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-066b1b73-7059-4388-a7dd-c71ceae401d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:07.418917 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1279/2561] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:07.430065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb192f4-300b-4022-b705-e94dde3d2e89 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.430423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb192f4-300b-4022-b705-e94dde3d2e89 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.430773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb192f4-300b-4022-b705-e94dde3d2e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:07.454095 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ccb192f4-300b-4022-b705-e94dde3d2e89 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:07.455394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ccb192f4-300b-4022-b705-e94dde3d2e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:07.464145 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1283/2562] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:07.468582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b1b73-7059-4388-a7dd-c71ceae401d5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:07.478637 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b1b73-7059-4388-a7dd-c71ceae401d5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:07.483960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296763d-ab0e-4141-8aa9-4712d894828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:07.484737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296763d-ab0e-4141-8aa9-4712d894828b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.485033 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296763d-ab0e-4141-8aa9-4712d894828b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.485296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296763d-ab0e-4141-8aa9-4712d894828b admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:07.486053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296763d-ab0e-4141-8aa9-4712d894828b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:09:07.486386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296763d-ab0e-4141-8aa9-4712d894828b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:07.488492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296763d-ab0e-4141-8aa9-4712d894828b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:07.501114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1280/2563] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:09:07.505694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb192f4-300b-4022-b705-e94dde3d2e89 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:07.516873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb192f4-300b-4022-b705-e94dde3d2e89 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:07.522107 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db134022-a219-4303-9168-f81fcf5682d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:07.522792 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db134022-a219-4303-9168-f81fcf5682d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.523073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db134022-a219-4303-9168-f81fcf5682d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.523349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db134022-a219-4303-9168-f81fcf5682d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:07.524069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db134022-a219-4303-9168-f81fcf5682d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:07.526032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db134022-a219-4303-9168-f81fcf5682d1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:07.554414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1284/2564] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:09:07 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:07.558547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296763d-ab0e-4141-8aa9-4712d894828b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:07.569259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296763d-ab0e-4141-8aa9-4712d894828b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:07.619579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:07.627916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066f9b1-b63f-4d13-8c5a-8618258533a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:07.628672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066f9b1-b63f-4d13-8c5a-8618258533a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.629319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066f9b1-b63f-4d13-8c5a-8618258533a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.629690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066f9b1-b63f-4d13-8c5a-8618258533a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:07.630664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066f9b1-b63f-4d13-8c5a-8618258533a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:07.632943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066f9b1-b63f-4d13-8c5a-8618258533a9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:07.655363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1281/2565] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:09:07 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:07.659479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db134022-a219-4303-9168-f81fcf5682d1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:07.674118 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db134022-a219-4303-9168-f81fcf5682d1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:07.679667 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e1766-d019-4a77-8afc-4eaf10d4e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:07.680590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e1766-d019-4a77-8afc-4eaf10d4e3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.681123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e1766-d019-4a77-8afc-4eaf10d4e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.681399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e1766-d019-4a77-8afc-4eaf10d4e3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:07.682301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e1766-d019-4a77-8afc-4eaf10d4e3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:07.684215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e1766-d019-4a77-8afc-4eaf10d4e3a5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:07.700696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1285/2566] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:07.702583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066f9b1-b63f-4d13-8c5a-8618258533a9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:07.719970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066f9b1-b63f-4d13-8c5a-8618258533a9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:07.725367 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022998f-f60d-4f54-b95e-ce3abd4eabea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:07.726691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022998f-f60d-4f54-b95e-ce3abd4eabea admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.726976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022998f-f60d-4f54-b95e-ce3abd4eabea admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.727253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022998f-f60d-4f54-b95e-ce3abd4eabea admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:07.728009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022998f-f60d-4f54-b95e-ce3abd4eabea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:07.729991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022998f-f60d-4f54-b95e-ce3abd4eabea admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:07.749199 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1282/2567] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:07.750591 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e1766-d019-4a77-8afc-4eaf10d4e3a5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:07.762282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e1766-d019-4a77-8afc-4eaf10d4e3a5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:07.767256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ed6d7-e010-4b5c-b0f5-5afd503a67dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:07.768221 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ed6d7-e010-4b5c-b0f5-5afd503a67dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.768486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ed6d7-e010-4b5c-b0f5-5afd503a67dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.768758 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ed6d7-e010-4b5c-b0f5-5afd503a67dc admin admin] PATH_INFO: `/v3/projects/6e816000ca6044a289d6d4014fba940e/users/81eeba58b05c405dbbe80df30080d6c7/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:07.784770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ed6d7-e010-4b5c-b0f5-5afd503a67dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e816000ca6044a289d6d4014fba940e, user_id=81eeba58b05c405dbbe80df30080d6c7, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:07.788392 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ed6d7-e010-4b5c-b0f5-5afd503a67dc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:07.803973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1286/2568] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:09:07 2022] PUT /identity/v3/projects/6e816000ca6044a289d6d4014fba940e/users/81eeba58b05c405dbbe80df30080d6c7/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:07.809632 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd7f74-3601-4443-81b7-daeb92c6f599 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.809979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd7f74-3601-4443-81b7-daeb92c6f599 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.810404 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd7f74-3601-4443-81b7-daeb92c6f599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:07.831807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-55dd7f74-3601-4443-81b7-daeb92c6f599 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:07.831807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-55dd7f74-3601-4443-81b7-daeb92c6f599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:07.900375 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1283/2569] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:09:07.911032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ed6d7-e010-4b5c-b0f5-5afd503a67dc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:07.912954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f60ed6d7-e010-4b5c-b0f5-5afd503a67dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:07.957524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:07.992043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ed6d7-e010-4b5c-b0f5-5afd503a67dc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:07.996394 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55738c-d882-4f7d-b8bd-e6f5c7c1f8e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:07.997060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55738c-d882-4f7d-b8bd-e6f5c7c1f8e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:07.997350 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55738c-d882-4f7d-b8bd-e6f5c7c1f8e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:07.997622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55738c-d882-4f7d-b8bd-e6f5c7c1f8e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:08.010347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55738c-d882-4f7d-b8bd-e6f5c7c1f8e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:08.012455 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55738c-d882-4f7d-b8bd-e6f5c7c1f8e7 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:08.024823 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1287/2570] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:09:09.064855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d8542-7718-4bfe-a172-81c4fa3d5b02 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:09.065422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d8542-7718-4bfe-a172-81c4fa3d5b02 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:09.065422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d8542-7718-4bfe-a172-81c4fa3d5b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:09.094250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-424d8542-7718-4bfe-a172-81c4fa3d5b02 None None] MFA Rules not processed for user `81eeba58b05c405dbbe80df30080d6c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:09.095458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-424d8542-7718-4bfe-a172-81c4fa3d5b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:09.163187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1284/2571] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:09.167921 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55738c-d882-4f7d-b8bd-e6f5c7c1f8e7 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:09.179254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55738c-d882-4f7d-b8bd-e6f5c7c1f8e7 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:09.184747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44e58a-e02a-4d05-9a74-7f997aff8fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:09.185479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44e58a-e02a-4d05-9a74-7f997aff8fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:09.186195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44e58a-e02a-4d05-9a74-7f997aff8fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:09.186195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44e58a-e02a-4d05-9a74-7f997aff8fc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:09.186908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44e58a-e02a-4d05-9a74-7f997aff8fc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:09.188927 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44e58a-e02a-4d05-9a74-7f997aff8fc8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:09.220186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1288/2572] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:09:09 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:09.221482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d8542-7718-4bfe-a172-81c4fa3d5b02 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:09.233198 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d8542-7718-4bfe-a172-81c4fa3d5b02 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:09.289620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fa620-0522-449a-ac44-48a603a363af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:09.290301 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fa620-0522-449a-ac44-48a603a363af admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:09.290584 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fa620-0522-449a-ac44-48a603a363af admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:09.290877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fa620-0522-449a-ac44-48a603a363af admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:09.291592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fa620-0522-449a-ac44-48a603a363af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:09.293579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fa620-0522-449a-ac44-48a603a363af admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:09.302849 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:09.318009 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1285/2573] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:09:09 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:09.322226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44e58a-e02a-4d05-9a74-7f997aff8fc8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:09.332970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44e58a-e02a-4d05-9a74-7f997aff8fc8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:09.338468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81c2f0-1f17-4ba6-a4f0-98e66ac2630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:09.339131 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81c2f0-1f17-4ba6-a4f0-98e66ac2630c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:09.339406 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81c2f0-1f17-4ba6-a4f0-98e66ac2630c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:09.339676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81c2f0-1f17-4ba6-a4f0-98e66ac2630c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:09.340443 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81c2f0-1f17-4ba6-a4f0-98e66ac2630c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:09.342047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81c2f0-1f17-4ba6-a4f0-98e66ac2630c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:09.353340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1289/2574] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:09.357658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fa620-0522-449a-ac44-48a603a363af admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:09.368558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fa620-0522-449a-ac44-48a603a363af admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:09.374174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500c843-05c8-4604-b461-5f5ed60c208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:09.375060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500c843-05c8-4604-b461-5f5ed60c208f admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:09.375336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500c843-05c8-4604-b461-5f5ed60c208f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:09.375608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500c843-05c8-4604-b461-5f5ed60c208f admin admin] PATH_INFO: `/v3/projects/0523e82ef87d4aad9908da459eedb09b/users/82710cf61f2247e49b4efb95883285fb/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:09.393444 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500c843-05c8-4604-b461-5f5ed60c208f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0523e82ef87d4aad9908da459eedb09b, user_id=82710cf61f2247e49b4efb95883285fb, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:09.396138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500c843-05c8-4604-b461-5f5ed60c208f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:09.413488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1286/2575] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:09:09 2022] PUT /identity/v3/projects/0523e82ef87d4aad9908da459eedb09b/users/82710cf61f2247e49b4efb95883285fb/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:09.417705 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81c2f0-1f17-4ba6-a4f0-98e66ac2630c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:09.429296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81c2f0-1f17-4ba6-a4f0-98e66ac2630c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:09.491059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439af09b-890e-4248-b8d0-a2e23e1c5894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:09.491752 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439af09b-890e-4248-b8d0-a2e23e1c5894 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:09.492035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439af09b-890e-4248-b8d0-a2e23e1c5894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:09.492342 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439af09b-890e-4248-b8d0-a2e23e1c5894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:09.493076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439af09b-890e-4248-b8d0-a2e23e1c5894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:09.494675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439af09b-890e-4248-b8d0-a2e23e1c5894 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:09.505427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1290/2576] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:09.509891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500c843-05c8-4604-b461-5f5ed60c208f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:09.520701 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500c843-05c8-4604-b461-5f5ed60c208f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:09.525738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64397-b793-4151-9ff3-c509b7c0f4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:09.526633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64397-b793-4151-9ff3-c509b7c0f4cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:09.526901 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64397-b793-4151-9ff3-c509b7c0f4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:09.527174 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64397-b793-4151-9ff3-c509b7c0f4cf admin admin] PATH_INFO: `/v3/projects/0523e82ef87d4aad9908da459eedb09b/users/82710cf61f2247e49b4efb95883285fb/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:09.529273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e64397-b793-4151-9ff3-c509b7c0f4cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0523e82ef87d4aad9908da459eedb09b, user_id=82710cf61f2247e49b4efb95883285fb, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:09.531918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e64397-b793-4151-9ff3-c509b7c0f4cf admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:09.542634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1287/2577] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:09:09 2022] PUT /identity/v3/projects/0523e82ef87d4aad9908da459eedb09b/users/82710cf61f2247e49b4efb95883285fb/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:10.274749 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26903ef-91f4-4d91-9b73-70cd167e45a2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:10.275270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26903ef-91f4-4d91-9b73-70cd167e45a2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:10.275391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26903ef-91f4-4d91-9b73-70cd167e45a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:10.303132 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a26903ef-91f4-4d91-9b73-70cd167e45a2 None None] MFA Rules not processed for user `82710cf61f2247e49b4efb95883285fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:10.304388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a26903ef-91f4-4d91-9b73-70cd167e45a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:10.377481 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1291/2578] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:10 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:11.515778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64397-b793-4151-9ff3-c509b7c0f4cf admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:11.589870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64397-b793-4151-9ff3-c509b7c0f4cf admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:11.595324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:11.596282 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:11.596558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:11.596827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] PATH_INFO: `/v3/users/81eeba58b05c405dbbe80df30080d6c7` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:11.598154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] RBAC: Authorizing `identity:delete_user(user_id=81eeba58b05c405dbbe80df30080d6c7)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:11.600253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:11.648207 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81eeba58b05c405dbbe80df30080d6c7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:11.648859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81eeba58b05c405dbbe80df30080d6c7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:11.655924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81eeba58b05c405dbbe80df30080d6c7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:11.656675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81eeba58b05c405dbbe80df30080d6c7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:11.670604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81eeba58b05c405dbbe80df30080d6c7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:11.686148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1288/2579] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:09:11 2022] DELETE /identity/v3/users/81eeba58b05c405dbbe80df30080d6c7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:11.867876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26903ef-91f4-4d91-9b73-70cd167e45a2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:11.880077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26903ef-91f4-4d91-9b73-70cd167e45a2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:11.903700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:11.950246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:11.951529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:11.951812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:11.952256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] PATH_INFO: `/v3/projects/6e816000ca6044a289d6d4014fba940e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:11.953503 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e816000ca6044a289d6d4014fba940e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:11.955576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:12.006427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e816000ca6044a289d6d4014fba940e', 'request_id': 'req-ed868c54-f6fe-42c0-8f43-b055bded9f8c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.012760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e816000ca6044a289d6d4014fba940e', 'request_id': 'req-ed868c54-f6fe-42c0-8f43-b055bded9f8c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.022161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e816000ca6044a289d6d4014fba940e', 'request_id': 'req-ed868c54-f6fe-42c0-8f43-b055bded9f8c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.032611 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] The token cache is being invalidate because project 6e816000ca6044a289d6d4014fba940e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:09:12.032895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.033504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.035166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1292/2580] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:09:11 2022] DELETE /identity/v3/projects/6e816000ca6044a289d6d4014fba940e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:12.039264 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:12.040726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:12.054471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a8178-0a12-4d3a-9f88-a9370d72495b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:12.087398 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:12.113848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:12.114652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:12.114931 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:12.115203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] PATH_INFO: `/v3/users/82710cf61f2247e49b4efb95883285fb` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:12.116450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82710cf61f2247e49b4efb95883285fb)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:12.118374 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:12.171818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82710cf61f2247e49b4efb95883285fb'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.172916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82710cf61f2247e49b4efb95883285fb'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.180394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82710cf61f2247e49b4efb95883285fb'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.181058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82710cf61f2247e49b4efb95883285fb'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.193135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-3156685b-5396-42db-941e-ed5372968ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82710cf61f2247e49b4efb95883285fb'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.207329 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1289/2581] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:09:12 2022] DELETE /identity/v3/users/82710cf61f2247e49b4efb95883285fb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:12.402364 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:12.404076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:12.417399 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868c54-f6fe-42c0-8f43-b055bded9f8c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:12.477396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:12.478274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:12.478549 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:12.478820 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] PATH_INFO: `/v3/projects/0523e82ef87d4aad9908da459eedb09b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:12.480029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0523e82ef87d4aad9908da459eedb09b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:12.481996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:12.554893 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0523e82ef87d4aad9908da459eedb09b', 'request_id': 'req-80ff868b-6eec-4f25-a2a9-29886b6fd44e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.556515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:12.566317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0523e82ef87d4aad9908da459eedb09b', 'request_id': 'req-80ff868b-6eec-4f25-a2a9-29886b6fd44e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.576286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0523e82ef87d4aad9908da459eedb09b', 'request_id': 'req-80ff868b-6eec-4f25-a2a9-29886b6fd44e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.588181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] The token cache is being invalidate because project 0523e82ef87d4aad9908da459eedb09b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:09:12.588474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.589114 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-80ff868b-6eec-4f25-a2a9-29886b6fd44e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:12.590698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1293/2582] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:09:12 2022] DELETE /identity/v3/projects/0523e82ef87d4aad9908da459eedb09b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:12.596040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639dddc-5043-434d-ba4e-32cff8c85659 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:12.596556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639dddc-5043-434d-ba4e-32cff8c85659 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:12.596736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639dddc-5043-434d-ba4e-32cff8c85659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:12.615812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3639dddc-5043-434d-ba4e-32cff8c85659 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:12.617060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3639dddc-5043-434d-ba4e-32cff8c85659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:12.681148 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1290/2583] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:12.689105 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b56b8e-cd27-419b-be64-6033d8117c3e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:12.689398 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b56b8e-cd27-419b-be64-6033d8117c3e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:12.689677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b56b8e-cd27-419b-be64-6033d8117c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:12.711728 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-59b56b8e-cd27-419b-be64-6033d8117c3e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:12.712984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-59b56b8e-cd27-419b-be64-6033d8117c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:12.722459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1294/2584] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:12.727413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639dddc-5043-434d-ba4e-32cff8c85659 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:12.738335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639dddc-5043-434d-ba4e-32cff8c85659 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:12.744015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c763c-8866-4d59-8e70-9b1ea3fc0b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:12.744697 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c763c-8866-4d59-8e70-9b1ea3fc0b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:12.744975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c763c-8866-4d59-8e70-9b1ea3fc0b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:12.745238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c763c-8866-4d59-8e70-9b1ea3fc0b0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:12.745979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c763c-8866-4d59-8e70-9b1ea3fc0b0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:09:12.746285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c763c-8866-4d59-8e70-9b1ea3fc0b0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:12.747877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c763c-8866-4d59-8e70-9b1ea3fc0b0e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:12.762396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1291/2585] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:09:12.767418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b56b8e-cd27-419b-be64-6033d8117c3e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:12.778352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b56b8e-cd27-419b-be64-6033d8117c3e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:12.783911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ddf27-df58-4712-b0c9-3e71469132db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:12.784637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ddf27-df58-4712-b0c9-3e71469132db admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:12.785063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ddf27-df58-4712-b0c9-3e71469132db admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:12.785370 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ddf27-df58-4712-b0c9-3e71469132db admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:12.786062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ddf27-df58-4712-b0c9-3e71469132db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:12.787976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ddf27-df58-4712-b0c9-3e71469132db admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:12.822673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1295/2586] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:09:12 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:12.822895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c763c-8866-4d59-8e70-9b1ea3fc0b0e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:12.839086 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c763c-8866-4d59-8e70-9b1ea3fc0b0e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:12.885997 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:12.910080 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5880e-1c0b-4854-9819-518d5ecbbc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:12.910856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5880e-1c0b-4854-9819-518d5ecbbc46 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:12.911543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5880e-1c0b-4854-9819-518d5ecbbc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:12.916633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5880e-1c0b-4854-9819-518d5ecbbc46 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:12.917650 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5880e-1c0b-4854-9819-518d5ecbbc46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:12.919376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5880e-1c0b-4854-9819-518d5ecbbc46 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:12.948100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1292/2587] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:09:12 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:12.949352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ddf27-df58-4712-b0c9-3e71469132db admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:12.960850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ddf27-df58-4712-b0c9-3e71469132db admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:12.966801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cd95c-0a32-4e5e-9401-e1310802949f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:12.967566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cd95c-0a32-4e5e-9401-e1310802949f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:12.967964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cd95c-0a32-4e5e-9401-e1310802949f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:12.968344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cd95c-0a32-4e5e-9401-e1310802949f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:12.969165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76cd95c-0a32-4e5e-9401-e1310802949f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:12.970885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76cd95c-0a32-4e5e-9401-e1310802949f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:12.985002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1296/2588] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:12.990177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5880e-1c0b-4854-9819-518d5ecbbc46 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:13.000488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5880e-1c0b-4854-9819-518d5ecbbc46 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:13.006218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cda06-93b8-42c0-8dab-b069b72e50b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:13.006930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cda06-93b8-42c0-8dab-b069b72e50b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.007305 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cda06-93b8-42c0-8dab-b069b72e50b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.007731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cda06-93b8-42c0-8dab-b069b72e50b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.008747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cda06-93b8-42c0-8dab-b069b72e50b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:13.010380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cda06-93b8-42c0-8dab-b069b72e50b8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:13.026627 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1293/2589] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:13.027815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cd95c-0a32-4e5e-9401-e1310802949f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:13.038234 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cd95c-0a32-4e5e-9401-e1310802949f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:13.042767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b905b35-7567-461d-a47d-5479a53213a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:13.043655 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b905b35-7567-461d-a47d-5479a53213a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.043930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b905b35-7567-461d-a47d-5479a53213a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.044231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b905b35-7567-461d-a47d-5479a53213a3 admin admin] PATH_INFO: `/v3/projects/2e0a1c18cd794160b2a8258218155fc7/users/8f055d8be5fe4fe99796de91129d6a1f/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.060102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b905b35-7567-461d-a47d-5479a53213a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0a1c18cd794160b2a8258218155fc7, user_id=8f055d8be5fe4fe99796de91129d6a1f, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:13.062964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b905b35-7567-461d-a47d-5479a53213a3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:13.078814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1297/2590] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:09:13 2022] PUT /identity/v3/projects/2e0a1c18cd794160b2a8258218155fc7/users/8f055d8be5fe4fe99796de91129d6a1f/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:09:13.090891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f8bb2-02b9-4fe3-9083-6941ef4c5326 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.091185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f8bb2-02b9-4fe3-9083-6941ef4c5326 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.091468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f8bb2-02b9-4fe3-9083-6941ef4c5326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.118712 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7a7f8bb2-02b9-4fe3-9083-6941ef4c5326 None None] MFA Rules not processed for user `8f055d8be5fe4fe99796de91129d6a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:13.119932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7a7f8bb2-02b9-4fe3-9083-6941ef4c5326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:13.191621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1294/2591] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:13.197047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b905b35-7567-461d-a47d-5479a53213a3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:13.269208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b905b35-7567-461d-a47d-5479a53213a3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:13.274489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f73d7-052f-48ed-b709-89a8e5fba3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:13.275149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f73d7-052f-48ed-b709-89a8e5fba3ea admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.275428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f73d7-052f-48ed-b709-89a8e5fba3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.275696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f73d7-052f-48ed-b709-89a8e5fba3ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.276408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f73d7-052f-48ed-b709-89a8e5fba3ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:13.278326 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f73d7-052f-48ed-b709-89a8e5fba3ea admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:13.310034 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1298/2592] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:09:13 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:13.313823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f8bb2-02b9-4fe3-9083-6941ef4c5326 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:13.325758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f8bb2-02b9-4fe3-9083-6941ef4c5326 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:13.385390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243e28e-f546-4b3d-aa39-d563b1c7da38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:13.386064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e28e-f546-4b3d-aa39-d563b1c7da38 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.386366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e28e-f546-4b3d-aa39-d563b1c7da38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.386640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e28e-f546-4b3d-aa39-d563b1c7da38 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.387331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243e28e-f546-4b3d-aa39-d563b1c7da38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:13.389627 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243e28e-f546-4b3d-aa39-d563b1c7da38 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:13.412175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1295/2593] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:09:13 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:13.416193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f73d7-052f-48ed-b709-89a8e5fba3ea admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:13.427333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f73d7-052f-48ed-b709-89a8e5fba3ea admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:13.433442 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbedd1-2b85-45e5-9753-8201571d88ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:13.434141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbedd1-2b85-45e5-9753-8201571d88ac admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.434450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbedd1-2b85-45e5-9753-8201571d88ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.434720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbedd1-2b85-45e5-9753-8201571d88ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.435452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbedd1-2b85-45e5-9753-8201571d88ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:13.437216 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbedd1-2b85-45e5-9753-8201571d88ac admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:13.448554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1299/2594] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:13.452874 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243e28e-f546-4b3d-aa39-d563b1c7da38 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:13.464355 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243e28e-f546-4b3d-aa39-d563b1c7da38 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:13.470241 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ac35f-06bb-481d-96b2-d0ef8831b7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:13.471142 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ac35f-06bb-481d-96b2-d0ef8831b7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.471421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ac35f-06bb-481d-96b2-d0ef8831b7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.471694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ac35f-06bb-481d-96b2-d0ef8831b7c1 admin admin] PATH_INFO: `/v3/projects/427bf411bb494dcf9e3bd7fa14dcb75a/users/b14aacb619024e2e83da3286b9895639/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.488692 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ac35f-06bb-481d-96b2-d0ef8831b7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=427bf411bb494dcf9e3bd7fa14dcb75a, user_id=b14aacb619024e2e83da3286b9895639, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:13.491351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ac35f-06bb-481d-96b2-d0ef8831b7c1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:13.506362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1296/2595] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:09:13 2022] PUT /identity/v3/projects/427bf411bb494dcf9e3bd7fa14dcb75a/users/b14aacb619024e2e83da3286b9895639/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:13.510514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbedd1-2b85-45e5-9753-8201571d88ac admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:13.520814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbedd1-2b85-45e5-9753-8201571d88ac admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:13.568991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:13.583725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5346fdb-2ce0-4f25-9677-0266f9c389b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:13.584440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5346fdb-2ce0-4f25-9677-0266f9c389b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.584720 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5346fdb-2ce0-4f25-9677-0266f9c389b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.584995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5346fdb-2ce0-4f25-9677-0266f9c389b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.585730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5346fdb-2ce0-4f25-9677-0266f9c389b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:13.587328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5346fdb-2ce0-4f25-9677-0266f9c389b9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:13.598066 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1300/2596] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:13.602386 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ac35f-06bb-481d-96b2-d0ef8831b7c1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:13.612969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ac35f-06bb-481d-96b2-d0ef8831b7c1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:13.618059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e68f9-2e53-41ea-a1ad-fbd52aae643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:13.618956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e68f9-2e53-41ea-a1ad-fbd52aae643f admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.619229 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e68f9-2e53-41ea-a1ad-fbd52aae643f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.619496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e68f9-2e53-41ea-a1ad-fbd52aae643f admin admin] PATH_INFO: `/v3/projects/427bf411bb494dcf9e3bd7fa14dcb75a/users/b14aacb619024e2e83da3286b9895639/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.621536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e68f9-2e53-41ea-a1ad-fbd52aae643f admin admin] RBAC: Authorizing `identity:create_grant(project_id=427bf411bb494dcf9e3bd7fa14dcb75a, user_id=b14aacb619024e2e83da3286b9895639, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:13.624196 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e68f9-2e53-41ea-a1ad-fbd52aae643f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:13.636463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1297/2597] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:09:13 2022] PUT /identity/v3/projects/427bf411bb494dcf9e3bd7fa14dcb75a/users/b14aacb619024e2e83da3286b9895639/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:13.646954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48b1f4-5bb7-4e08-ba28-d008e95cc539 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.647347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48b1f4-5bb7-4e08-ba28-d008e95cc539 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.647461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48b1f4-5bb7-4e08-ba28-d008e95cc539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.680244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7b48b1f4-5bb7-4e08-ba28-d008e95cc539 None None] MFA Rules not processed for user `b14aacb619024e2e83da3286b9895639`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:13.681452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7b48b1f4-5bb7-4e08-ba28-d008e95cc539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:13.756051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1301/2598] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:13.767516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e68f9-2e53-41ea-a1ad-fbd52aae643f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:13.769471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c56e68f9-2e53-41ea-a1ad-fbd52aae643f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:13.775061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:13.832923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e68f9-2e53-41ea-a1ad-fbd52aae643f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:13.837528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43508fe3-430e-4b74-b117-eb36e79129b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:13.838189 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43508fe3-430e-4b74-b117-eb36e79129b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:13.838494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43508fe3-430e-4b74-b117-eb36e79129b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:13.838767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43508fe3-430e-4b74-b117-eb36e79129b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:13.898317 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43508fe3-430e-4b74-b117-eb36e79129b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:13.899767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43508fe3-430e-4b74-b117-eb36e79129b2 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:13.908966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1298/2599] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:09:14.972042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48b1f4-5bb7-4e08-ba28-d008e95cc539 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:14.974845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7b48b1f4-5bb7-4e08-ba28-d008e95cc539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:15.059089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48b1f4-5bb7-4e08-ba28-d008e95cc539 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:15.064783 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afe1af-f672-424f-b907-5f98551c5310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:15.065436 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afe1af-f672-424f-b907-5f98551c5310 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:15.065718 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afe1af-f672-424f-b907-5f98551c5310 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:15.065990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afe1af-f672-424f-b907-5f98551c5310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:15.067797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b6afe1af-f672-424f-b907-5f98551c5310 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:15.079734 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afe1af-f672-424f-b907-5f98551c5310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:15.081388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afe1af-f672-424f-b907-5f98551c5310 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:15.091802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1302/2600] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:09:21.650000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43508fe3-430e-4b74-b117-eb36e79129b2 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:21.651678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-43508fe3-430e-4b74-b117-eb36e79129b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:21.668161 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43508fe3-430e-4b74-b117-eb36e79129b2 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:21.668884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00df02-d0ad-442e-ab8f-2ad3297154b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:21.669943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00df02-d0ad-442e-ab8f-2ad3297154b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:21.670248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00df02-d0ad-442e-ab8f-2ad3297154b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:21.670608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00df02-d0ad-442e-ab8f-2ad3297154b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:21.681254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00df02-d0ad-442e-ab8f-2ad3297154b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:21.682705 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00df02-d0ad-442e-ab8f-2ad3297154b3 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:21.692503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1299/2601] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:09:25.087622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=75 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:30.624636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8a5f9-e526-4520-8c3b-133b58a723fa None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:30.625199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8a5f9-e526-4520-8c3b-133b58a723fa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:30.625199 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8a5f9-e526-4520-8c3b-133b58a723fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:30.680547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d0e8a5f9-e526-4520-8c3b-133b58a723fa None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:30.681923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d0e8a5f9-e526-4520-8c3b-133b58a723fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:30.751496 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1303/2602] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 21:09:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:30.799201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00df02-d0ad-442e-ab8f-2ad3297154b3 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:30.800786 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7f00df02-d0ad-442e-ab8f-2ad3297154b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:30.899085 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00df02-d0ad-442e-ab8f-2ad3297154b3 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:30.906323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0025fb3-ab21-433e-9d0f-3893dce9b5de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:30.906978 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0025fb3-ab21-433e-9d0f-3893dce9b5de service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:30.907257 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0025fb3-ab21-433e-9d0f-3893dce9b5de service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:30.907526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0025fb3-ab21-433e-9d0f-3893dce9b5de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:30.918462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0025fb3-ab21-433e-9d0f-3893dce9b5de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:30.920091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0025fb3-ab21-433e-9d0f-3893dce9b5de service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:30.928739 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1300/2603] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:09:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:09:31.449895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50443dd-81e6-4636-9de4-6c45a67b2e07 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:31.450195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50443dd-81e6-4636-9de4-6c45a67b2e07 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:31.450465 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50443dd-81e6-4636-9de4-6c45a67b2e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:31.468815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f50443dd-81e6-4636-9de4-6c45a67b2e07 None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:31.470006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f50443dd-81e6-4636-9de4-6c45a67b2e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:31.477729 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1304/2604] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jul 11 21:09:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:31.489926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0025fb3-ab21-433e-9d0f-3893dce9b5de service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:31.500918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0025fb3-ab21-433e-9d0f-3893dce9b5de service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:31.506319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51237356-889f-4010-aa2e-07f8c1bf45ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:31.506956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51237356-889f-4010-aa2e-07f8c1bf45ff service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:31.507234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51237356-889f-4010-aa2e-07f8c1bf45ff service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:31.507513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51237356-889f-4010-aa2e-07f8c1bf45ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:31.516192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51237356-889f-4010-aa2e-07f8c1bf45ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:31.517785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51237356-889f-4010-aa2e-07f8c1bf45ff service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:31.522406 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:31.527865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1301/2605] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:09:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:09:31.736644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61a967-f680-4cdc-a48b-6cbb61925ee2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:31.737150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61a967-f680-4cdc-a48b-6cbb61925ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:31.737150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61a967-f680-4cdc-a48b-6cbb61925ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:31.760130 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0e61a967-f680-4cdc-a48b-6cbb61925ee2 None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:31.761738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0e61a967-f680-4cdc-a48b-6cbb61925ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:31.770255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1305/2606] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jul 11 21:09:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:31.782376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51237356-889f-4010-aa2e-07f8c1bf45ff service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:31.793579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51237356-889f-4010-aa2e-07f8c1bf45ff service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:31.799183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dfc18-666d-4a9a-90e9-5912f061040e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:31.799814 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dfc18-666d-4a9a-90e9-5912f061040e service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:31.800405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dfc18-666d-4a9a-90e9-5912f061040e service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:31.800405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dfc18-666d-4a9a-90e9-5912f061040e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:31.808530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3dfc18-666d-4a9a-90e9-5912f061040e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:31.810138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3dfc18-666d-4a9a-90e9-5912f061040e service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:31.818240 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1302/2607] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:09:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:09:32.151213 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522321b-6de4-4213-b2ee-c95d125ed6a2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:32.151515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522321b-6de4-4213-b2ee-c95d125ed6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:32.151886 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522321b-6de4-4213-b2ee-c95d125ed6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:32.169986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8522321b-6de4-4213-b2ee-c95d125ed6a2 None None] MFA Rules not processed for user `812f2c4a42fd470c9c57bd14310f320f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:32.171190 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8522321b-6de4-4213-b2ee-c95d125ed6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:32.179082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1306/2608] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jul 11 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:32.194579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dfc18-666d-4a9a-90e9-5912f061040e service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:32.210691 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dfc18-666d-4a9a-90e9-5912f061040e service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:32.217764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809be2c1-8944-46e3-9026-ed396a15369b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97b5f28d84fe4570a7c5c904dc4e502c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:32.218413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be2c1-8944-46e3-9026-ed396a15369b service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:32.218688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be2c1-8944-46e3-9026-ed396a15369b service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:32.218956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be2c1-8944-46e3-9026-ed396a15369b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:32.227576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809be2c1-8944-46e3-9026-ed396a15369b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:32.229190 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809be2c1-8944-46e3-9026-ed396a15369b service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:32.237748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1303/2609] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jul 11 21:09:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 21:09:35.609489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522321b-6de4-4213-b2ee-c95d125ed6a2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:35.611129 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8522321b-6de4-4213-b2ee-c95d125ed6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:35.679547 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522321b-6de4-4213-b2ee-c95d125ed6a2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:35.685155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:35.686004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:35.686279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:35.686548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] PATH_INFO: `/v3/users/fe0be3efded64c04ad05b984b52c534f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:35.687759 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0be3efded64c04ad05b984b52c534f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:35.689747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:35.739918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0be3efded64c04ad05b984b52c534f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:35.745357 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:35.747438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0be3efded64c04ad05b984b52c534f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:35.748266 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0be3efded64c04ad05b984b52c534f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:35.756569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0be3efded64c04ad05b984b52c534f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:35.768532 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0be3efded64c04ad05b984b52c534f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:35.774620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1307/2610] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:09:35 2022] DELETE /identity/v3/users/fe0be3efded64c04ad05b984b52c534f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:35.779422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca0f7-d586-4fe8-a5a1-2caea406f9f6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:35.779922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca0f7-d586-4fe8-a5a1-2caea406f9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:35.780019 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca0f7-d586-4fe8-a5a1-2caea406f9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:35.798218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1e8ca0f7-d586-4fe8-a5a1-2caea406f9f6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:35.799416 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1e8ca0f7-d586-4fe8-a5a1-2caea406f9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:35.863443 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1304/2611] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:09:35.873710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:35.875170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:35.934119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeff8c4-62d1-4d53-890b-7974b6e7f2d7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:35.938055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c97f83-32a9-4fab-9435-f3ba6b6690e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:35.938714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97f83-32a9-4fab-9435-f3ba6b6690e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:35.938995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97f83-32a9-4fab-9435-f3ba6b6690e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:35.939273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97f83-32a9-4fab-9435-f3ba6b6690e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:35.948472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c97f83-32a9-4fab-9435-f3ba6b6690e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:35.949940 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c97f83-32a9-4fab-9435-f3ba6b6690e9 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:35.960562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1308/2612] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:09:36.220211 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ca0f7-d586-4fe8-a5a1-2caea406f9f6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:36.221745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1e8ca0f7-d586-4fe8-a5a1-2caea406f9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:36.234220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ca0f7-d586-4fe8-a5a1-2caea406f9f6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:36.239937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:36.240868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.241144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.241461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] PATH_INFO: `/v3/projects/3539cabdbfa943a49368f866270a776b` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:36.242818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3539cabdbfa943a49368f866270a776b)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:36.244807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:36.316389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3539cabdbfa943a49368f866270a776b', 'request_id': 'req-ad6e03fa-9618-48d8-97f2-35890dd07fa3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:36.323759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3539cabdbfa943a49368f866270a776b', 'request_id': 'req-ad6e03fa-9618-48d8-97f2-35890dd07fa3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:36.334859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3539cabdbfa943a49368f866270a776b', 'request_id': 'req-ad6e03fa-9618-48d8-97f2-35890dd07fa3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:36.351024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] The token cache is being invalidate because project 3539cabdbfa943a49368f866270a776b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:09:36.351313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:36.351991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad6e03fa-9618-48d8-97f2-35890dd07fa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:36.353636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1305/2613] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:09:36 2022] DELETE /identity/v3/projects/3539cabdbfa943a49368f866270a776b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:36.359102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e599b0e-77d9-4f02-bee2-6583953bfaa6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.359685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e599b0e-77d9-4f02-bee2-6583953bfaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.359964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e599b0e-77d9-4f02-bee2-6583953bfaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:36.378129 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0e599b0e-77d9-4f02-bee2-6583953bfaa6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:36.379305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0e599b0e-77d9-4f02-bee2-6583953bfaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:36.443649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1309/2614] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:36.451303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7e932-0db5-4677-a5ab-c963a8b4a3e8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.451597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7e932-0db5-4677-a5ab-c963a8b4a3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.451878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7e932-0db5-4677-a5ab-c963a8b4a3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:36.470889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-16d7e932-0db5-4677-a5ab-c963a8b4a3e8 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:36.472256 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-16d7e932-0db5-4677-a5ab-c963a8b4a3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:36.480582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1306/2615] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:36.484878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e599b0e-77d9-4f02-bee2-6583953bfaa6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:36.494608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e599b0e-77d9-4f02-bee2-6583953bfaa6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:36.499487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244c13e-9cbf-48d3-9e7d-48760213fd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:36.500172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244c13e-9cbf-48d3-9e7d-48760213fd40 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.500453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244c13e-9cbf-48d3-9e7d-48760213fd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.500737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244c13e-9cbf-48d3-9e7d-48760213fd40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:36.501468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244c13e-9cbf-48d3-9e7d-48760213fd40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:09:36.501769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244c13e-9cbf-48d3-9e7d-48760213fd40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:36.503385 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244c13e-9cbf-48d3-9e7d-48760213fd40 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:36.518509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1310/2616] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:09:36.519818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7e932-0db5-4677-a5ab-c963a8b4a3e8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:36.529771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7e932-0db5-4677-a5ab-c963a8b4a3e8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:36.534741 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8963f6-79e9-4fc0-9eba-374ca3f3f93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:36.535407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8963f6-79e9-4fc0-9eba-374ca3f3f93b admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.535695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8963f6-79e9-4fc0-9eba-374ca3f3f93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.535958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8963f6-79e9-4fc0-9eba-374ca3f3f93b admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:36.536675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8963f6-79e9-4fc0-9eba-374ca3f3f93b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:36.538589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8963f6-79e9-4fc0-9eba-374ca3f3f93b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:36.566635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1307/2617] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:09:36 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:36.570872 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244c13e-9cbf-48d3-9e7d-48760213fd40 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:36.581833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244c13e-9cbf-48d3-9e7d-48760213fd40 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:36.644524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149f3ff-eae1-42bf-921b-530a13f828e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:36.645219 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149f3ff-eae1-42bf-921b-530a13f828e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.645485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149f3ff-eae1-42bf-921b-530a13f828e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.645757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149f3ff-eae1-42bf-921b-530a13f828e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:36.646459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149f3ff-eae1-42bf-921b-530a13f828e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:36.648445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149f3ff-eae1-42bf-921b-530a13f828e7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:36.657124 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:36.671801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1311/2618] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:09:36 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:09:36.676341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8963f6-79e9-4fc0-9eba-374ca3f3f93b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:36.687482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8963f6-79e9-4fc0-9eba-374ca3f3f93b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:36.693415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed5e9f-0acd-41e9-a16d-ced2651fdead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:36.694324 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed5e9f-0acd-41e9-a16d-ced2651fdead admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.694474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed5e9f-0acd-41e9-a16d-ced2651fdead admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.694761 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed5e9f-0acd-41e9-a16d-ced2651fdead admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:36.695561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed5e9f-0acd-41e9-a16d-ced2651fdead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:36.697225 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed5e9f-0acd-41e9-a16d-ced2651fdead admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:36.708620 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1308/2619] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:36.712625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149f3ff-eae1-42bf-921b-530a13f828e7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:36.723024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149f3ff-eae1-42bf-921b-530a13f828e7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:36.727699 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb3b3b-9b1d-47cf-8a0e-1852bf1ce558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:36.728403 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb3b3b-9b1d-47cf-8a0e-1852bf1ce558 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.728693 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb3b3b-9b1d-47cf-8a0e-1852bf1ce558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.728966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb3b3b-9b1d-47cf-8a0e-1852bf1ce558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:36.729726 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb3b3b-9b1d-47cf-8a0e-1852bf1ce558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:36.731317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb3b3b-9b1d-47cf-8a0e-1852bf1ce558 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:36.741679 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1312/2620] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:09:36.745814 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed5e9f-0acd-41e9-a16d-ced2651fdead admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:36.756512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed5e9f-0acd-41e9-a16d-ced2651fdead admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:36.761744 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3371e-b3ae-4dd8-82b9-a51480fdb23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:36.762646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3371e-b3ae-4dd8-82b9-a51480fdb23c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.763037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3371e-b3ae-4dd8-82b9-a51480fdb23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.763191 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3371e-b3ae-4dd8-82b9-a51480fdb23c admin admin] PATH_INFO: `/v3/projects/e70de8bb8790476f94e064c645d8c147/users/64b677fcf3da48339d152bdbe8ff435b/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:36.779457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3371e-b3ae-4dd8-82b9-a51480fdb23c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70de8bb8790476f94e064c645d8c147, user_id=64b677fcf3da48339d152bdbe8ff435b, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:36.782082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3371e-b3ae-4dd8-82b9-a51480fdb23c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:36.784343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:36.798767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1309/2621] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:09:36 2022] PUT /identity/v3/projects/e70de8bb8790476f94e064c645d8c147/users/64b677fcf3da48339d152bdbe8ff435b/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:36.809202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac869f-aa69-4b6c-8627-2f8b4bcdd175 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.809499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac869f-aa69-4b6c-8627-2f8b4bcdd175 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.809807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac869f-aa69-4b6c-8627-2f8b4bcdd175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:36.838366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-14ac869f-aa69-4b6c-8627-2f8b4bcdd175 None None] MFA Rules not processed for user `64b677fcf3da48339d152bdbe8ff435b`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:36.839603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-14ac869f-aa69-4b6c-8627-2f8b4bcdd175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:36.911476 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1313/2622] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:36.924894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3371e-b3ae-4dd8-82b9-a51480fdb23c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:36.926909 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bfe3371e-b3ae-4dd8-82b9-a51480fdb23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:36.990490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3371e-b3ae-4dd8-82b9-a51480fdb23c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:36.995710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2353d0c-6c66-4b37-ba55-961e76d03808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:36.996389 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2353d0c-6c66-4b37-ba55-961e76d03808 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:36.996683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2353d0c-6c66-4b37-ba55-961e76d03808 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:36.996955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2353d0c-6c66-4b37-ba55-961e76d03808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:37.005967 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2353d0c-6c66-4b37-ba55-961e76d03808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:37.007413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2353d0c-6c66-4b37-ba55-961e76d03808 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:37.017235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1310/2623] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:09:46.907120 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:47.012843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:55.559305 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac869f-aa69-4b6c-8627-2f8b4bcdd175 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:55.625439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac869f-aa69-4b6c-8627-2f8b4bcdd175 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:55.635060 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:55.636302 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:55.636590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:55.636871 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] PATH_INFO: `/v3/users/64b677fcf3da48339d152bdbe8ff435b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:55.638186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b677fcf3da48339d152bdbe8ff435b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:55.640274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:55.693971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b677fcf3da48339d152bdbe8ff435b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:55.699011 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b677fcf3da48339d152bdbe8ff435b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:55.699751 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b677fcf3da48339d152bdbe8ff435b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:55.708687 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b677fcf3da48339d152bdbe8ff435b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:55.723949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b677fcf3da48339d152bdbe8ff435b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:55.731584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1314/2624] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:09:55 2022] DELETE /identity/v3/users/64b677fcf3da48339d152bdbe8ff435b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:55.736754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5721-dbea-4e72-86fb-951a178f710d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:55.737748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5721-dbea-4e72-86fb-951a178f710d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:55.738034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5721-dbea-4e72-86fb-951a178f710d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:55.755561 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e12f5721-dbea-4e72-86fb-951a178f710d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:55.756804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e12f5721-dbea-4e72-86fb-951a178f710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:55.836681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1311/2625] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:09:55.837920 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:55.840056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:55.898311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f996e-64cf-4bc7-bd41-da33689f8d90 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:55.902514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fa35f-9e1d-439f-bcb9-a8f6651068fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:55.903158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fa35f-9e1d-439f-bcb9-a8f6651068fd service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:55.903439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fa35f-9e1d-439f-bcb9-a8f6651068fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:55.903708 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fa35f-9e1d-439f-bcb9-a8f6651068fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:55.913108 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fa35f-9e1d-439f-bcb9-a8f6651068fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:55.914541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fa35f-9e1d-439f-bcb9-a8f6651068fd service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:55.925278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1315/2626] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 21:09:56.209583 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f5721-dbea-4e72-86fb-951a178f710d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:56.211662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e12f5721-dbea-4e72-86fb-951a178f710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:56.225535 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f5721-dbea-4e72-86fb-951a178f710d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:56.231539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:56.232589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:56.232872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:56.233145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] PATH_INFO: `/v3/projects/e70de8bb8790476f94e064c645d8c147` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:56.234388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70de8bb8790476f94e064c645d8c147)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:56.236459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:56.290633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70de8bb8790476f94e064c645d8c147', 'request_id': 'req-b05d7e18-adcc-485d-994e-ef6f472fe109'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:56.297541 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70de8bb8790476f94e064c645d8c147', 'request_id': 'req-b05d7e18-adcc-485d-994e-ef6f472fe109'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:56.306851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70de8bb8790476f94e064c645d8c147', 'request_id': 'req-b05d7e18-adcc-485d-994e-ef6f472fe109'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:56.318516 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] The token cache is being invalidate because project e70de8bb8790476f94e064c645d8c147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:09:56.318808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:56.319445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b05d7e18-adcc-485d-994e-ef6f472fe109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:56.321066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1312/2627] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:09:56 2022] DELETE /identity/v3/projects/e70de8bb8790476f94e064c645d8c147 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:09:58.138021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fa35f-9e1d-439f-bcb9-a8f6651068fd service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:58.139644 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e01fa35f-9e1d-439f-bcb9-a8f6651068fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:58.153657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fa35f-9e1d-439f-bcb9-a8f6651068fd service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:58.179794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:58.223884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:58.224808 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:58.225084 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:58.225352 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] PATH_INFO: `/v3/users/8f055d8be5fe4fe99796de91129d6a1f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:58.226567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f055d8be5fe4fe99796de91129d6a1f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:58.228557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:58.278248 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f055d8be5fe4fe99796de91129d6a1f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:58.285245 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f055d8be5fe4fe99796de91129d6a1f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:58.285903 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f055d8be5fe4fe99796de91129d6a1f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:58.294568 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f055d8be5fe4fe99796de91129d6a1f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:58.307285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f055d8be5fe4fe99796de91129d6a1f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:58.313296 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1316/2628] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:09:58 2022] DELETE /identity/v3/users/8f055d8be5fe4fe99796de91129d6a1f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:58.318182 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb9c08-ea21-47e0-9854-07a653fbcfc9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:58.318719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb9c08-ea21-47e0-9854-07a653fbcfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:58.319040 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb9c08-ea21-47e0-9854-07a653fbcfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:58.339472 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-75eb9c08-ea21-47e0-9854-07a653fbcfc9 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:58.340744 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-75eb9c08-ea21-47e0-9854-07a653fbcfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:58.410891 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1313/2629] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:58.422869 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:58.424567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:58.484141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e982ad4-4417-4337-8dd1-e2337002fb5e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:58.488310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d47872-774c-4a5c-8ecf-cd163300db88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:58.488957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d47872-774c-4a5c-8ecf-cd163300db88 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:58.489233 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d47872-774c-4a5c-8ecf-cd163300db88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:58.489507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d47872-774c-4a5c-8ecf-cd163300db88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:58.498859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d47872-774c-4a5c-8ecf-cd163300db88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:58.500333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d47872-774c-4a5c-8ecf-cd163300db88 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:58.509768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1317/2630] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:09:58.788323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb9c08-ea21-47e0-9854-07a653fbcfc9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:58.789872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-75eb9c08-ea21-47e0-9854-07a653fbcfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:58.802787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb9c08-ea21-47e0-9854-07a653fbcfc9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:58.809227 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:58.810089 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:58.810387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:58.810646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] PATH_INFO: `/v3/projects/2e0a1c18cd794160b2a8258218155fc7` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:58.811859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0a1c18cd794160b2a8258218155fc7)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:58.813801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:58.837457 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:58.870380 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0a1c18cd794160b2a8258218155fc7', 'request_id': 'req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:58.876808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0a1c18cd794160b2a8258218155fc7', 'request_id': 'req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:58.885723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0a1c18cd794160b2a8258218155fc7', 'request_id': 'req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:58.900246 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] The token cache is being invalidate because project 2e0a1c18cd794160b2a8258218155fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:09:58.900710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:58.901736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:58.904774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1314/2631] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:09:58 2022] DELETE /identity/v3/projects/2e0a1c18cd794160b2a8258218155fc7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:58.909159 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d47872-774c-4a5c-8ecf-cd163300db88 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:58.920297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d47872-774c-4a5c-8ecf-cd163300db88 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:58.967906 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:09:58.989112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:58.990420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:58.990804 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:58.991076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] PATH_INFO: `/v3/users/b14aacb619024e2e83da3286b9895639` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:58.992957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b14aacb619024e2e83da3286b9895639)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:58.995535 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:59.049151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b14aacb619024e2e83da3286b9895639'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:59.056416 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b14aacb619024e2e83da3286b9895639'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:59.057149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b14aacb619024e2e83da3286b9895639'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:59.067581 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b14aacb619024e2e83da3286b9895639'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:59.088029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-82662dc1-c070-46e6-958b-f2ff02a46ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b14aacb619024e2e83da3286b9895639'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:59.096185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1318/2632] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:09:58 2022] DELETE /identity/v3/users/b14aacb619024e2e83da3286b9895639 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:59.666461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:59.669545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:59.684408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee42b2-d5d1-4fd1-b4c8-80cc2739f30c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:59.743520 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:59.744354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:59.744895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:59.745185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] PATH_INFO: `/v3/projects/427bf411bb494dcf9e3bd7fa14dcb75a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:59.746416 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=427bf411bb494dcf9e3bd7fa14dcb75a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:59.748372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:59.797492 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427bf411bb494dcf9e3bd7fa14dcb75a', 'request_id': 'req-e6b6f359-c266-46d4-bd95-c05acbdb8a79'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:59.803579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427bf411bb494dcf9e3bd7fa14dcb75a', 'request_id': 'req-e6b6f359-c266-46d4-bd95-c05acbdb8a79'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:59.812175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427bf411bb494dcf9e3bd7fa14dcb75a', 'request_id': 'req-e6b6f359-c266-46d4-bd95-c05acbdb8a79'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:59.823248 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] The token cache is being invalidate because project 427bf411bb494dcf9e3bd7fa14dcb75a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:09:59.823544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:59.824215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e6b6f359-c266-46d4-bd95-c05acbdb8a79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:09:59.825848 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1315/2633] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:09:59 2022] DELETE /identity/v3/projects/427bf411bb494dcf9e3bd7fa14dcb75a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:09:59.830996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8cf7c-6a7f-44a6-a8f7-8eb7b5ba0c82 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:59.831496 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8cf7c-6a7f-44a6-a8f7-8eb7b5ba0c82 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:59.831603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8cf7c-6a7f-44a6-a8f7-8eb7b5ba0c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:59.850529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f8e8cf7c-6a7f-44a6-a8f7-8eb7b5ba0c82 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:59.851743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f8e8cf7c-6a7f-44a6-a8f7-8eb7b5ba0c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:59.916116 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1319/2634] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:59.925231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa35fb-a483-453b-9dc3-f0ec66336356 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:59.925517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa35fb-a483-453b-9dc3-f0ec66336356 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:59.925789 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa35fb-a483-453b-9dc3-f0ec66336356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:59.945136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-abfa35fb-a483-453b-9dc3-f0ec66336356 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:09:59.946345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-abfa35fb-a483-453b-9dc3-f0ec66336356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:09:59.955539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1316/2635] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:09:59.960214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8cf7c-6a7f-44a6-a8f7-8eb7b5ba0c82 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:09:59.971412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8cf7c-6a7f-44a6-a8f7-8eb7b5ba0c82 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:09:59.978039 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c27be-dc9d-4b5e-8832-a02897b3b7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:09:59.978737 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c27be-dc9d-4b5e-8832-a02897b3b7df admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:09:59.979156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c27be-dc9d-4b5e-8832-a02897b3b7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:09:59.979433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c27be-dc9d-4b5e-8832-a02897b3b7df admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:09:59.980178 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c27be-dc9d-4b5e-8832-a02897b3b7df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:09:59.980513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c27be-dc9d-4b5e-8832-a02897b3b7df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:09:59.982137 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c27be-dc9d-4b5e-8832-a02897b3b7df admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:09:59.994504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1320/2636] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:09:59.998955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa35fb-a483-453b-9dc3-f0ec66336356 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.008513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa35fb-a483-453b-9dc3-f0ec66336356 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.013602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4feb54-a9a1-47ef-b0c8-018896b25247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.014255 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4feb54-a9a1-47ef-b0c8-018896b25247 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.014536 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4feb54-a9a1-47ef-b0c8-018896b25247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.014811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4feb54-a9a1-47ef-b0c8-018896b25247 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.015502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4feb54-a9a1-47ef-b0c8-018896b25247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.017437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4feb54-a9a1-47ef-b0c8-018896b25247 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.045177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1317/2637] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:09:59 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:10:00.049739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c27be-dc9d-4b5e-8832-a02897b3b7df admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.060478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c27be-dc9d-4b5e-8832-a02897b3b7df admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.108537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:00.124226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e13fa1-14f2-4a32-bd03-3450e91ccba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.124907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e13fa1-14f2-4a32-bd03-3450e91ccba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.125191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e13fa1-14f2-4a32-bd03-3450e91ccba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.125468 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e13fa1-14f2-4a32-bd03-3450e91ccba0 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.126162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e13fa1-14f2-4a32-bd03-3450e91ccba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.128144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e13fa1-14f2-4a32-bd03-3450e91ccba0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.149266 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1321/2638] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:10:00 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:10:00.153894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4feb54-a9a1-47ef-b0c8-018896b25247 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.165203 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4feb54-a9a1-47ef-b0c8-018896b25247 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.171327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc32b4-1d94-4249-8fba-82b2d3c21cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.172350 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc32b4-1d94-4249-8fba-82b2d3c21cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.172659 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc32b4-1d94-4249-8fba-82b2d3c21cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.172925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc32b4-1d94-4249-8fba-82b2d3c21cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.173658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc32b4-1d94-4249-8fba-82b2d3c21cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.175235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc32b4-1d94-4249-8fba-82b2d3c21cc8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.177510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:00.191926 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1318/2639] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:10:00.196412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e13fa1-14f2-4a32-bd03-3450e91ccba0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.212099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e13fa1-14f2-4a32-bd03-3450e91ccba0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.218291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a661ea6-9c36-49d1-b9c8-2eb4a58fca26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.218960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a661ea6-9c36-49d1-b9c8-2eb4a58fca26 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.219347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a661ea6-9c36-49d1-b9c8-2eb4a58fca26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.219599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a661ea6-9c36-49d1-b9c8-2eb4a58fca26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.220382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a661ea6-9c36-49d1-b9c8-2eb4a58fca26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.221970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a661ea6-9c36-49d1-b9c8-2eb4a58fca26 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.232303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1322/2640] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:10:00.236757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc32b4-1d94-4249-8fba-82b2d3c21cc8 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.247347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc32b4-1d94-4249-8fba-82b2d3c21cc8 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.252838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5e907-a1d1-4ac7-87c3-0bfdb1d761d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.253759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5e907-a1d1-4ac7-87c3-0bfdb1d761d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.254035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5e907-a1d1-4ac7-87c3-0bfdb1d761d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.254311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5e907-a1d1-4ac7-87c3-0bfdb1d761d2 admin admin] PATH_INFO: `/v3/projects/a3e2af19b130444c99f5a39824b33c78/users/2c31b30488e54e65add68a72171a955b/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.270200 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5e907-a1d1-4ac7-87c3-0bfdb1d761d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e2af19b130444c99f5a39824b33c78, user_id=2c31b30488e54e65add68a72171a955b, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.272889 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5e907-a1d1-4ac7-87c3-0bfdb1d761d2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.288124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1319/2641] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:10:00 2022] PUT /identity/v3/projects/a3e2af19b130444c99f5a39824b33c78/users/2c31b30488e54e65add68a72171a955b/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:00.299746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4368-c712-42ce-805d-808c780ca9db None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.300047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4368-c712-42ce-805d-808c780ca9db None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.300445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4368-c712-42ce-805d-808c780ca9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.332054 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c6ad4368-c712-42ce-805d-808c780ca9db None None] MFA Rules not processed for user `2c31b30488e54e65add68a72171a955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:00.333313 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c6ad4368-c712-42ce-805d-808c780ca9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:00.400664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1323/2642] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:00.405556 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5e907-a1d1-4ac7-87c3-0bfdb1d761d2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.482482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5e907-a1d1-4ac7-87c3-0bfdb1d761d2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.487524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc689e65-6f3d-49cc-9070-f31c5d6a6f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.488219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc689e65-6f3d-49cc-9070-f31c5d6a6f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.488500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc689e65-6f3d-49cc-9070-f31c5d6a6f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.488775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc689e65-6f3d-49cc-9070-f31c5d6a6f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.489471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc689e65-6f3d-49cc-9070-f31c5d6a6f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.491403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc689e65-6f3d-49cc-9070-f31c5d6a6f5b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.519378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1320/2643] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:10:00 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:10:00.523658 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4368-c712-42ce-805d-808c780ca9db None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.534791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4368-c712-42ce-805d-808c780ca9db None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.597643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395bdc5-08e9-4f81-b6ff-e95184922445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.598573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395bdc5-08e9-4f81-b6ff-e95184922445 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.598941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395bdc5-08e9-4f81-b6ff-e95184922445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.599240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395bdc5-08e9-4f81-b6ff-e95184922445 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.600277 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395bdc5-08e9-4f81-b6ff-e95184922445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.603459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395bdc5-08e9-4f81-b6ff-e95184922445 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.626750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1324/2644] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:10:00 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:10:00.630826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc689e65-6f3d-49cc-9070-f31c5d6a6f5b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.641862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc689e65-6f3d-49cc-9070-f31c5d6a6f5b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.647162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb403b9a-35e6-48f8-a3be-b1d28807bb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.647822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb403b9a-35e6-48f8-a3be-b1d28807bb0f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.648124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb403b9a-35e6-48f8-a3be-b1d28807bb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.648410 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb403b9a-35e6-48f8-a3be-b1d28807bb0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.649141 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb403b9a-35e6-48f8-a3be-b1d28807bb0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.650743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb403b9a-35e6-48f8-a3be-b1d28807bb0f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.662039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1321/2645] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:10:00.666277 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395bdc5-08e9-4f81-b6ff-e95184922445 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.678214 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395bdc5-08e9-4f81-b6ff-e95184922445 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.683768 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3b895-569a-48e8-a308-8b17c9c643ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.684703 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3b895-569a-48e8-a308-8b17c9c643ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.684987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3b895-569a-48e8-a308-8b17c9c643ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.685265 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3b895-569a-48e8-a308-8b17c9c643ce admin admin] PATH_INFO: `/v3/projects/861d9f532e1a471ebe7b3cfda07b0ae9/users/a086a1c90ed849eb9e7d7a33b2739eac/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.701170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3b895-569a-48e8-a308-8b17c9c643ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=861d9f532e1a471ebe7b3cfda07b0ae9, user_id=a086a1c90ed849eb9e7d7a33b2739eac, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.703827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3b895-569a-48e8-a308-8b17c9c643ce admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.723211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1325/2646] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:10:00 2022] PUT /identity/v3/projects/861d9f532e1a471ebe7b3cfda07b0ae9/users/a086a1c90ed849eb9e7d7a33b2739eac/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:00.727459 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb403b9a-35e6-48f8-a3be-b1d28807bb0f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.738449 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb403b9a-35e6-48f8-a3be-b1d28807bb0f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.798501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541bec4-2777-41da-8834-57f6ca418299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.799083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541bec4-2777-41da-8834-57f6ca418299 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.799363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541bec4-2777-41da-8834-57f6ca418299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.799644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541bec4-2777-41da-8834-57f6ca418299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.800388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541bec4-2777-41da-8834-57f6ca418299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.801984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541bec4-2777-41da-8834-57f6ca418299 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.812490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1322/2647] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:10:00.816974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3b895-569a-48e8-a308-8b17c9c643ce admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.828834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3b895-569a-48e8-a308-8b17c9c643ce admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:00.834530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07d5d-f1f4-4849-9d83-480d96c3346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:00.835378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07d5d-f1f4-4849-9d83-480d96c3346f admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.835660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07d5d-f1f4-4849-9d83-480d96c3346f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.835941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07d5d-f1f4-4849-9d83-480d96c3346f admin admin] PATH_INFO: `/v3/projects/861d9f532e1a471ebe7b3cfda07b0ae9/users/a086a1c90ed849eb9e7d7a33b2739eac/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.838152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a07d5d-f1f4-4849-9d83-480d96c3346f admin admin] RBAC: Authorizing `identity:create_grant(project_id=861d9f532e1a471ebe7b3cfda07b0ae9, user_id=a086a1c90ed849eb9e7d7a33b2739eac, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:00.840967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a07d5d-f1f4-4849-9d83-480d96c3346f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:00.852061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1326/2648] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:10:00 2022] PUT /identity/v3/projects/861d9f532e1a471ebe7b3cfda07b0ae9/users/a086a1c90ed849eb9e7d7a33b2739eac/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:00.862469 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ffbfd-afdc-4691-8b80-4a2464c503e9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:00.862755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ffbfd-afdc-4691-8b80-4a2464c503e9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:00.863015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ffbfd-afdc-4691-8b80-4a2464c503e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:00.889969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-715ffbfd-afdc-4691-8b80-4a2464c503e9 None None] MFA Rules not processed for user `a086a1c90ed849eb9e7d7a33b2739eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:00.891193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-715ffbfd-afdc-4691-8b80-4a2464c503e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:00.926551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:00.969234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1323/2649] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:00.981382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07d5d-f1f4-4849-9d83-480d96c3346f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:00.983498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-69a07d5d-f1f4-4849-9d83-480d96c3346f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:00.999092 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:01.045845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07d5d-f1f4-4849-9d83-480d96c3346f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:01.050439 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab10c1d-f61a-4c82-83c4-f0b355f32375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:01.051015 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab10c1d-f61a-4c82-83c4-f0b355f32375 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:01.051306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab10c1d-f61a-4c82-83c4-f0b355f32375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:01.051577 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab10c1d-f61a-4c82-83c4-f0b355f32375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:01.061544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab10c1d-f61a-4c82-83c4-f0b355f32375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:01.063147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab10c1d-f61a-4c82-83c4-f0b355f32375 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:01.072710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1327/2650] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:10:01.158832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ffbfd-afdc-4691-8b80-4a2464c503e9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:01.169566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ffbfd-afdc-4691-8b80-4a2464c503e9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:01.174221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5fcf4-dba8-4741-8656-82af89244ac8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:01.174878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5fcf4-dba8-4741-8656-82af89244ac8 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:01.175162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5fcf4-dba8-4741-8656-82af89244ac8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:01.175463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5fcf4-dba8-4741-8656-82af89244ac8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:01.235356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5fcf4-dba8-4741-8656-82af89244ac8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:01.237048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5fcf4-dba8-4741-8656-82af89244ac8 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:01.251592 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1324/2651] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:10:06.089812 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab10c1d-f61a-4c82-83c4-f0b355f32375 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:06.154827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab10c1d-f61a-4c82-83c4-f0b355f32375 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:06.159764 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:06.160687 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:06.160968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:06.161242 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] PATH_INFO: `/v3/users/2c31b30488e54e65add68a72171a955b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:06.162469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c31b30488e54e65add68a72171a955b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:06.164452 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:06.214211 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c31b30488e54e65add68a72171a955b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.221206 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c31b30488e54e65add68a72171a955b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.221854 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c31b30488e54e65add68a72171a955b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.230732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c31b30488e54e65add68a72171a955b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.244271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c31b30488e54e65add68a72171a955b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.250723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1328/2652] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:10:06 2022] DELETE /identity/v3/users/2c31b30488e54e65add68a72171a955b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:06.255695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a610b4e-107d-437e-9455-93b2f1138d3d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:06.256223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a610b4e-107d-437e-9455-93b2f1138d3d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:06.256365 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a610b4e-107d-437e-9455-93b2f1138d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:06.274209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4a610b4e-107d-437e-9455-93b2f1138d3d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:06.275418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4a610b4e-107d-437e-9455-93b2f1138d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:06.335372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1325/2653] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:06.345574 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:06.347109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:06.384854 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:06.407527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4ce6a-c2a8-4b60-9ed6-dd31615fa953 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:06.411279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc871af7-f12b-47ca-95d4-ff55730eac15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:06.411925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc871af7-f12b-47ca-95d4-ff55730eac15 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:06.412237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc871af7-f12b-47ca-95d4-ff55730eac15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:06.412510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc871af7-f12b-47ca-95d4-ff55730eac15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:06.421756 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc871af7-f12b-47ca-95d4-ff55730eac15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:06.423659 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc871af7-f12b-47ca-95d4-ff55730eac15 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:06.433220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1329/2654] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:10:06.725022 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a610b4e-107d-437e-9455-93b2f1138d3d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:06.726552 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4a610b4e-107d-437e-9455-93b2f1138d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:06.743118 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a610b4e-107d-437e-9455-93b2f1138d3d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:06.749230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:06.750383 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:06.750716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:06.750925 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] PATH_INFO: `/v3/projects/a3e2af19b130444c99f5a39824b33c78` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:06.752300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e2af19b130444c99f5a39824b33c78)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:06.754261 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:06.814054 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e2af19b130444c99f5a39824b33c78', 'request_id': 'req-116a9d5c-d175-44f7-92a7-174ea3ea547e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.820688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e2af19b130444c99f5a39824b33c78', 'request_id': 'req-116a9d5c-d175-44f7-92a7-174ea3ea547e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.830138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e2af19b130444c99f5a39824b33c78', 'request_id': 'req-116a9d5c-d175-44f7-92a7-174ea3ea547e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.832378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:06.844405 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] The token cache is being invalidate because project a3e2af19b130444c99f5a39824b33c78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:10:06.844652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.845321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.846932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1326/2655] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:10:06 2022] DELETE /identity/v3/projects/a3e2af19b130444c99f5a39824b33c78 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:06.851195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc871af7-f12b-47ca-95d4-ff55730eac15 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:06.861150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc871af7-f12b-47ca-95d4-ff55730eac15 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:06.920433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:06.921312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:06.921589 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:06.921853 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] PATH_INFO: `/v3/users/a086a1c90ed849eb9e7d7a33b2739eac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:06.923529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a086a1c90ed849eb9e7d7a33b2739eac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:06.925492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:06.973347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a086a1c90ed849eb9e7d7a33b2739eac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.980233 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a086a1c90ed849eb9e7d7a33b2739eac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.980878 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a086a1c90ed849eb9e7d7a33b2739eac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:06.988875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a086a1c90ed849eb9e7d7a33b2739eac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:07.000588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e5f938bf-7ab7-4290-a3d2-541eca075361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a086a1c90ed849eb9e7d7a33b2739eac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:07.006393 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1330/2656] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:10:06 2022] DELETE /identity/v3/users/a086a1c90ed849eb9e7d7a33b2739eac => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:10:07.280274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:07.282662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:07.295893 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a9d5c-d175-44f7-92a7-174ea3ea547e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:07.353999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:07.354870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:07.355147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:07.355417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] PATH_INFO: `/v3/projects/861d9f532e1a471ebe7b3cfda07b0ae9` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:07.356635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=861d9f532e1a471ebe7b3cfda07b0ae9)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:07.358534 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:07.407230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861d9f532e1a471ebe7b3cfda07b0ae9', 'request_id': 'req-48100b61-531f-419e-977f-d10b35bf9cc5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:07.415391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861d9f532e1a471ebe7b3cfda07b0ae9', 'request_id': 'req-48100b61-531f-419e-977f-d10b35bf9cc5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:07.424296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861d9f532e1a471ebe7b3cfda07b0ae9', 'request_id': 'req-48100b61-531f-419e-977f-d10b35bf9cc5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:07.435187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] The token cache is being invalidate because project 861d9f532e1a471ebe7b3cfda07b0ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:10:07.435470 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:07.436085 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-48100b61-531f-419e-977f-d10b35bf9cc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:07.437744 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1327/2657] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:10:07 2022] DELETE /identity/v3/projects/861d9f532e1a471ebe7b3cfda07b0ae9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:07.442868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711603e5-a3a3-40ae-9e81-b9e3c5a4ddae None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:07.443345 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711603e5-a3a3-40ae-9e81-b9e3c5a4ddae None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:07.443443 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711603e5-a3a3-40ae-9e81-b9e3c5a4ddae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:07.462584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-711603e5-a3a3-40ae-9e81-b9e3c5a4ddae None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:07.463794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-711603e5-a3a3-40ae-9e81-b9e3c5a4ddae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:07.515838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:07.529089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1331/2658] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:07.537059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bc2ec-0fa5-4ca8-9bc8-eeae43e5410d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:07.537378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bc2ec-0fa5-4ca8-9bc8-eeae43e5410d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:07.537654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bc2ec-0fa5-4ca8-9bc8-eeae43e5410d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:07.557961 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ab0bc2ec-0fa5-4ca8-9bc8-eeae43e5410d None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:07.559178 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ab0bc2ec-0fa5-4ca8-9bc8-eeae43e5410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:07.568529 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1328/2659] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:07.573454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711603e5-a3a3-40ae-9e81-b9e3c5a4ddae None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:07.589047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711603e5-a3a3-40ae-9e81-b9e3c5a4ddae None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:07.596514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ab885-637b-429a-b0c1-d2689b50597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:07.597172 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab885-637b-429a-b0c1-d2689b50597a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:07.597450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab885-637b-429a-b0c1-d2689b50597a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:07.597722 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab885-637b-429a-b0c1-d2689b50597a admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:07.598453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ab885-637b-429a-b0c1-d2689b50597a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:10:07.598757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ab885-637b-429a-b0c1-d2689b50597a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:07.600431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ab885-637b-429a-b0c1-d2689b50597a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:07.612141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1332/2660] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:10:07.616417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0bc2ec-0fa5-4ca8-9bc8-eeae43e5410d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:07.626384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0bc2ec-0fa5-4ca8-9bc8-eeae43e5410d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:07.632707 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886c937-eea7-4080-8d58-ae26e17606d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:07.633363 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886c937-eea7-4080-8d58-ae26e17606d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:07.633644 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886c937-eea7-4080-8d58-ae26e17606d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:07.633916 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886c937-eea7-4080-8d58-ae26e17606d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:07.634652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886c937-eea7-4080-8d58-ae26e17606d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:07.636611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886c937-eea7-4080-8d58-ae26e17606d9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:07.668177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1329/2661] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:10:07 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:10:07.673673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ab885-637b-429a-b0c1-d2689b50597a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:07.689005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ab885-637b-429a-b0c1-d2689b50597a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:07.760233 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c22e3-700d-42c2-8852-04ab20830b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:07.760918 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c22e3-700d-42c2-8852-04ab20830b3f admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:07.761243 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c22e3-700d-42c2-8852-04ab20830b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:07.761507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c22e3-700d-42c2-8852-04ab20830b3f admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:07.762209 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c22e3-700d-42c2-8852-04ab20830b3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:07.764184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c22e3-700d-42c2-8852-04ab20830b3f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:07.785519 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1333/2662] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:10:07 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:10:07.789875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886c937-eea7-4080-8d58-ae26e17606d9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:07.801210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886c937-eea7-4080-8d58-ae26e17606d9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:07.806636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa9307-1620-4fbb-97eb-5f4a30f2df5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:07.807300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa9307-1620-4fbb-97eb-5f4a30f2df5b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:07.807577 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa9307-1620-4fbb-97eb-5f4a30f2df5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:07.807845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa9307-1620-4fbb-97eb-5f4a30f2df5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:07.808611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa9307-1620-4fbb-97eb-5f4a30f2df5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:07.810217 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa9307-1620-4fbb-97eb-5f4a30f2df5b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:07.821615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1330/2663] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:10:07.825894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c22e3-700d-42c2-8852-04ab20830b3f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:07.836781 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c22e3-700d-42c2-8852-04ab20830b3f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:07.842810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a4ff4-635d-4932-b49d-6c2d861ac2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:07.843499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a4ff4-635d-4932-b49d-6c2d861ac2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:07.843782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a4ff4-635d-4932-b49d-6c2d861ac2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:07.844223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a4ff4-635d-4932-b49d-6c2d861ac2e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:07.844973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a4ff4-635d-4932-b49d-6c2d861ac2e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:07.846583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a4ff4-635d-4932-b49d-6c2d861ac2e2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:07.856911 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1334/2664] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:10:07.861895 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa9307-1620-4fbb-97eb-5f4a30f2df5b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:07.877051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa9307-1620-4fbb-97eb-5f4a30f2df5b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:07.882401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0c6d9-047b-4ca4-b1a3-e733f2b843c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:07.883303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0c6d9-047b-4ca4-b1a3-e733f2b843c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:07.883582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0c6d9-047b-4ca4-b1a3-e733f2b843c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:07.883856 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0c6d9-047b-4ca4-b1a3-e733f2b843c6 admin admin] PATH_INFO: `/v3/projects/ec0b1e651da043f1becc96ebfd3302ed/users/a936e2c759b14c86aacae128768caed7/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:07.899250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0c6d9-047b-4ca4-b1a3-e733f2b843c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0b1e651da043f1becc96ebfd3302ed, user_id=a936e2c759b14c86aacae128768caed7, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:07.901958 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0c6d9-047b-4ca4-b1a3-e733f2b843c6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:07.921500 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1331/2665] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:10:07 2022] PUT /identity/v3/projects/ec0b1e651da043f1becc96ebfd3302ed/users/a936e2c759b14c86aacae128768caed7/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:07.932590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c43746-753b-47fa-a04f-aaf2fe3945a1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:07.932850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c43746-753b-47fa-a04f-aaf2fe3945a1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:07.933162 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c43746-753b-47fa-a04f-aaf2fe3945a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:07.961727 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-83c43746-753b-47fa-a04f-aaf2fe3945a1 None None] MFA Rules not processed for user `a936e2c759b14c86aacae128768caed7`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:07.962956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-83c43746-753b-47fa-a04f-aaf2fe3945a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:08.033002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1335/2666] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:08.037541 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0c6d9-047b-4ca4-b1a3-e733f2b843c6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:08.058519 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:08.106453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0c6d9-047b-4ca4-b1a3-e733f2b843c6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:08.115238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80818760-2c2d-4d1a-a4d8-ffa31ea0a350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:08.115939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80818760-2c2d-4d1a-a4d8-ffa31ea0a350 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:08.116260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80818760-2c2d-4d1a-a4d8-ffa31ea0a350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:08.116537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80818760-2c2d-4d1a-a4d8-ffa31ea0a350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:08.117237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80818760-2c2d-4d1a-a4d8-ffa31ea0a350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:08.119166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80818760-2c2d-4d1a-a4d8-ffa31ea0a350 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:08.153230 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1332/2667] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:10:08 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:10:08.157673 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c43746-753b-47fa-a04f-aaf2fe3945a1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:08.169562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c43746-753b-47fa-a04f-aaf2fe3945a1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:08.220464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:08.234711 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74143c03-6c7a-4a3e-9ff4-66c24322bfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:08.235396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74143c03-6c7a-4a3e-9ff4-66c24322bfa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:08.235719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74143c03-6c7a-4a3e-9ff4-66c24322bfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:08.235954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74143c03-6c7a-4a3e-9ff4-66c24322bfa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:08.236714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74143c03-6c7a-4a3e-9ff4-66c24322bfa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:08.238642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74143c03-6c7a-4a3e-9ff4-66c24322bfa3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:08.260038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1336/2668] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:10:08 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:10:08.264855 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80818760-2c2d-4d1a-a4d8-ffa31ea0a350 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:08.275647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80818760-2c2d-4d1a-a4d8-ffa31ea0a350 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:08.280713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1432759-008b-407a-81cb-326993bdd4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:08.281371 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1432759-008b-407a-81cb-326993bdd4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:08.282231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1432759-008b-407a-81cb-326993bdd4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:08.282540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1432759-008b-407a-81cb-326993bdd4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:08.283273 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1432759-008b-407a-81cb-326993bdd4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:08.284907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1432759-008b-407a-81cb-326993bdd4c3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:08.295777 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1333/2669] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:10:08.300240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74143c03-6c7a-4a3e-9ff4-66c24322bfa3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:08.310348 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74143c03-6c7a-4a3e-9ff4-66c24322bfa3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:08.315430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2207089-2631-4496-8bcc-33d74fc7f460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:08.316344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2207089-2631-4496-8bcc-33d74fc7f460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:08.316619 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2207089-2631-4496-8bcc-33d74fc7f460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:08.316890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2207089-2631-4496-8bcc-33d74fc7f460 admin admin] PATH_INFO: `/v3/projects/b67be534cee34148b1a3883a90e67bc6/users/1bab2e52627543a8afd0641c15ed810e/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:08.336280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2207089-2631-4496-8bcc-33d74fc7f460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67be534cee34148b1a3883a90e67bc6, user_id=1bab2e52627543a8afd0641c15ed810e, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:08.338949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2207089-2631-4496-8bcc-33d74fc7f460 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:08.354306 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1337/2670] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:10:08 2022] PUT /identity/v3/projects/b67be534cee34148b1a3883a90e67bc6/users/1bab2e52627543a8afd0641c15ed810e/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:08.358565 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1432759-008b-407a-81cb-326993bdd4c3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:08.370404 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1432759-008b-407a-81cb-326993bdd4c3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:08.444506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cad52a-121a-4689-80c0-fe6f2504dd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:08.445482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad52a-121a-4689-80c0-fe6f2504dd16 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:08.445942 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad52a-121a-4689-80c0-fe6f2504dd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:08.446407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad52a-121a-4689-80c0-fe6f2504dd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:08.447566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cad52a-121a-4689-80c0-fe6f2504dd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:08.450732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cad52a-121a-4689-80c0-fe6f2504dd16 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:08.466594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1334/2671] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:10:08.471234 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2207089-2631-4496-8bcc-33d74fc7f460 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:08.482520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2207089-2631-4496-8bcc-33d74fc7f460 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:08.488483 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc90c25-c713-4782-bf42-cfb96ecd4ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:08.489389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc90c25-c713-4782-bf42-cfb96ecd4ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:08.489665 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc90c25-c713-4782-bf42-cfb96ecd4ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:08.489938 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc90c25-c713-4782-bf42-cfb96ecd4ac2 admin admin] PATH_INFO: `/v3/projects/b67be534cee34148b1a3883a90e67bc6/users/1bab2e52627543a8afd0641c15ed810e/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:08.492225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc90c25-c713-4782-bf42-cfb96ecd4ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67be534cee34148b1a3883a90e67bc6, user_id=1bab2e52627543a8afd0641c15ed810e, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:08.494896 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc90c25-c713-4782-bf42-cfb96ecd4ac2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:08.506215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1338/2672] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:10:08 2022] PUT /identity/v3/projects/b67be534cee34148b1a3883a90e67bc6/users/1bab2e52627543a8afd0641c15ed810e/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:08.517517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f066643-b925-409c-a2b3-67cce9967a3e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:08.517806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f066643-b925-409c-a2b3-67cce9967a3e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:08.518084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f066643-b925-409c-a2b3-67cce9967a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:08.546168 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1f066643-b925-409c-a2b3-67cce9967a3e None None] MFA Rules not processed for user `1bab2e52627543a8afd0641c15ed810e`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:08.547748 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1f066643-b925-409c-a2b3-67cce9967a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:08.617922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1335/2673] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:08.629178 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc90c25-c713-4782-bf42-cfb96ecd4ac2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:08.630985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-abc90c25-c713-4782-bf42-cfb96ecd4ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:08.691193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc90c25-c713-4782-bf42-cfb96ecd4ac2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:08.697839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f775f1-e8b9-4e81-b1d6-7be538000b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:08.698679 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f775f1-e8b9-4e81-b1d6-7be538000b83 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:08.699026 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f775f1-e8b9-4e81-b1d6-7be538000b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:08.699378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f775f1-e8b9-4e81-b1d6-7be538000b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:08.711051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f775f1-e8b9-4e81-b1d6-7be538000b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:08.712559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f775f1-e8b9-4e81-b1d6-7be538000b83 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:08.722840 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1339/2674] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:10:09.106453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f066643-b925-409c-a2b3-67cce9967a3e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:09.120579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:09.179339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f066643-b925-409c-a2b3-67cce9967a3e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:09.184440 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:09.185307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:09.185615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:09.185841 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] PATH_INFO: `/v3/users/a936e2c759b14c86aacae128768caed7` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:09.187115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a936e2c759b14c86aacae128768caed7)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:09.189233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:09.251360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a936e2c759b14c86aacae128768caed7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.251938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a936e2c759b14c86aacae128768caed7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.260306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a936e2c759b14c86aacae128768caed7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.261018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a936e2c759b14c86aacae128768caed7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.276223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a936e2c759b14c86aacae128768caed7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.291653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1336/2675] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:10:09 2022] DELETE /identity/v3/users/a936e2c759b14c86aacae128768caed7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:09.296558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87578ac6-0af5-47b8-ac5f-9e04e69c584e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:09.296849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87578ac6-0af5-47b8-ac5f-9e04e69c584e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:09.297471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87578ac6-0af5-47b8-ac5f-9e04e69c584e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:09.315741 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-87578ac6-0af5-47b8-ac5f-9e04e69c584e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:09.316983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-87578ac6-0af5-47b8-ac5f-9e04e69c584e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:09.376098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1340/2676] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:09.388699 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:09.390150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:09.449952 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ce18b-97be-41b3-88e8-dc6e620036fb admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:09.454158 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39119f-3fc4-47c7-bcb7-f3553709da97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:09.454808 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39119f-3fc4-47c7-bcb7-f3553709da97 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:09.455088 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39119f-3fc4-47c7-bcb7-f3553709da97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:09.455357 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39119f-3fc4-47c7-bcb7-f3553709da97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:09.464870 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39119f-3fc4-47c7-bcb7-f3553709da97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:09.466366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39119f-3fc4-47c7-bcb7-f3553709da97 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:09.476018 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1337/2677] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 21:10:09.757875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87578ac6-0af5-47b8-ac5f-9e04e69c584e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:09.759489 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-87578ac6-0af5-47b8-ac5f-9e04e69c584e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:09.775347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87578ac6-0af5-47b8-ac5f-9e04e69c584e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:09.780094 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:09.781142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:09.781620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:09.781949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] PATH_INFO: `/v3/projects/ec0b1e651da043f1becc96ebfd3302ed` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:09.783307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec0b1e651da043f1becc96ebfd3302ed)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:09.785257 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:09.832510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:09.835763 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0b1e651da043f1becc96ebfd3302ed', 'request_id': 'req-61fa0bce-796f-47b7-a260-b578c8859c37'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.843599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0b1e651da043f1becc96ebfd3302ed', 'request_id': 'req-61fa0bce-796f-47b7-a260-b578c8859c37'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.853274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0b1e651da043f1becc96ebfd3302ed', 'request_id': 'req-61fa0bce-796f-47b7-a260-b578c8859c37'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.864321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] The token cache is being invalidate because project ec0b1e651da043f1becc96ebfd3302ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:10:09.864610 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.865226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.866786 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1341/2678] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:10:09 2022] DELETE /identity/v3/projects/ec0b1e651da043f1becc96ebfd3302ed => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:09.871041 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39119f-3fc4-47c7-bcb7-f3553709da97 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:09.881474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39119f-3fc4-47c7-bcb7-f3553709da97 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:09.915025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:09.941732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:09.942602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:09.942887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:09.943155 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] PATH_INFO: `/v3/users/1bab2e52627543a8afd0641c15ed810e` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:09.944387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bab2e52627543a8afd0641c15ed810e)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:09.946307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:09.994066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bab2e52627543a8afd0641c15ed810e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:09.994727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bab2e52627543a8afd0641c15ed810e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:10.001950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bab2e52627543a8afd0641c15ed810e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:10.002588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bab2e52627543a8afd0641c15ed810e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:10.014249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-79f777a9-542a-465b-a2f3-faa9b0754c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bab2e52627543a8afd0641c15ed810e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:10.028270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1338/2679] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:10:09 2022] DELETE /identity/v3/users/1bab2e52627543a8afd0641c15ed810e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:10:10.310554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:10.312082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:10.325404 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa0bce-796f-47b7-a260-b578c8859c37 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:10.383504 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:10.384378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:10.384658 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:10.384934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] PATH_INFO: `/v3/projects/b67be534cee34148b1a3883a90e67bc6` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:10.386127 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b67be534cee34148b1a3883a90e67bc6)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:10.388075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:10.444084 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67be534cee34148b1a3883a90e67bc6', 'request_id': 'req-0cce4b7e-9cbb-437f-8c65-830e2f357323'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:10.451778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67be534cee34148b1a3883a90e67bc6', 'request_id': 'req-0cce4b7e-9cbb-437f-8c65-830e2f357323'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:10.462376 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67be534cee34148b1a3883a90e67bc6', 'request_id': 'req-0cce4b7e-9cbb-437f-8c65-830e2f357323'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:10.473743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] The token cache is being invalidate because project b67be534cee34148b1a3883a90e67bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:10:10.474049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:10.474700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0cce4b7e-9cbb-437f-8c65-830e2f357323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:10.476289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1342/2680] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:10:10 2022] DELETE /identity/v3/projects/b67be534cee34148b1a3883a90e67bc6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:10.482360 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427ca9f-da64-4dfb-a82d-b7e1306faa9b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:10.482879 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427ca9f-da64-4dfb-a82d-b7e1306faa9b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:10.482999 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427ca9f-da64-4dfb-a82d-b7e1306faa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:10.501981 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4427ca9f-da64-4dfb-a82d-b7e1306faa9b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:10.503185 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4427ca9f-da64-4dfb-a82d-b7e1306faa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:10.565869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1339/2681] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:10.574217 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd8b8c-46dd-4c4b-b595-f526ab57e536 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:10.574513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd8b8c-46dd-4c4b-b595-f526ab57e536 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:10.574795 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd8b8c-46dd-4c4b-b595-f526ab57e536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:10.596333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a9dd8b8c-46dd-4c4b-b595-f526ab57e536 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:10.597554 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a9dd8b8c-46dd-4c4b-b595-f526ab57e536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:10.607046 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1343/2682] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:10.612094 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427ca9f-da64-4dfb-a82d-b7e1306faa9b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:10.627017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427ca9f-da64-4dfb-a82d-b7e1306faa9b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:10.632007 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a442-e9d6-48d8-8396-90b0f7b47073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:10.632690 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a442-e9d6-48d8-8396-90b0f7b47073 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:10.632991 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a442-e9d6-48d8-8396-90b0f7b47073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:10.633236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a442-e9d6-48d8-8396-90b0f7b47073 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:10.634051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629a442-e9d6-48d8-8396-90b0f7b47073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:10:10.634290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629a442-e9d6-48d8-8396-90b0f7b47073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:10.635867 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629a442-e9d6-48d8-8396-90b0f7b47073 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:10.647700 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1340/2683] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:10:10.652269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd8b8c-46dd-4c4b-b595-f526ab57e536 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:10.662571 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd8b8c-46dd-4c4b-b595-f526ab57e536 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:10.668212 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5c2bc-6163-4964-8ddc-f67f04d14ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:10.668870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c2bc-6163-4964-8ddc-f67f04d14ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:10.669149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c2bc-6163-4964-8ddc-f67f04d14ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:10.669417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c2bc-6163-4964-8ddc-f67f04d14ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:10.670166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5c2bc-6163-4964-8ddc-f67f04d14ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:10.672538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5c2bc-6163-4964-8ddc-f67f04d14ab8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:10.705431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1344/2684] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:10:10 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:10:10.711506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a442-e9d6-48d8-8396-90b0f7b47073 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:10.722339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a442-e9d6-48d8-8396-90b0f7b47073 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:10.765078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:10.785236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff66d80-c6e7-4400-b0fd-fc7f4a9f6df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:10.786231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff66d80-c6e7-4400-b0fd-fc7f4a9f6df4 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:10.786568 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff66d80-c6e7-4400-b0fd-fc7f4a9f6df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:10.786869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff66d80-c6e7-4400-b0fd-fc7f4a9f6df4 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:10.787573 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff66d80-c6e7-4400-b0fd-fc7f4a9f6df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:10.789551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff66d80-c6e7-4400-b0fd-fc7f4a9f6df4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:10.813505 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1341/2685] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:10:10 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:10:10.818043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5c2bc-6163-4964-8ddc-f67f04d14ab8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:10.828948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5c2bc-6163-4964-8ddc-f67f04d14ab8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:10.834067 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dc33a-c58a-4739-9487-02bd283445ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:10.834805 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dc33a-c58a-4739-9487-02bd283445ac admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:10.835007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dc33a-c58a-4739-9487-02bd283445ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:10.835282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dc33a-c58a-4739-9487-02bd283445ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:10.836018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dc33a-c58a-4739-9487-02bd283445ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:10.837677 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dc33a-c58a-4739-9487-02bd283445ac admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:10.848771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1345/2686] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:10:10.853051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff66d80-c6e7-4400-b0fd-fc7f4a9f6df4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:10.863613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff66d80-c6e7-4400-b0fd-fc7f4a9f6df4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:10.868720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e97b0-86f9-49d7-a49a-2c0bc0e98222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:10.869376 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e97b0-86f9-49d7-a49a-2c0bc0e98222 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:10.869662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e97b0-86f9-49d7-a49a-2c0bc0e98222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:10.869939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e97b0-86f9-49d7-a49a-2c0bc0e98222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:10.870672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e97b0-86f9-49d7-a49a-2c0bc0e98222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:10.872310 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e97b0-86f9-49d7-a49a-2c0bc0e98222 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:10.882490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1342/2687] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:10:10.886695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dc33a-c58a-4739-9487-02bd283445ac admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:10.896983 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dc33a-c58a-4739-9487-02bd283445ac admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:10.902366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b415f89-30a7-4085-a7b3-75dcf29c3a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:10.903250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b415f89-30a7-4085-a7b3-75dcf29c3a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:10.903497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b415f89-30a7-4085-a7b3-75dcf29c3a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:10.903802 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b415f89-30a7-4085-a7b3-75dcf29c3a65 admin admin] PATH_INFO: `/v3/projects/6b688b7b65aa4b6d8d2ce55ccd3a9279/users/9567d59139a64afa82500c750e4406bb/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:10.919445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b415f89-30a7-4085-a7b3-75dcf29c3a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b688b7b65aa4b6d8d2ce55ccd3a9279, user_id=9567d59139a64afa82500c750e4406bb, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:10.922684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b415f89-30a7-4085-a7b3-75dcf29c3a65 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:10.925776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:10.940809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1346/2688] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:10:10 2022] PUT /identity/v3/projects/6b688b7b65aa4b6d8d2ce55ccd3a9279/users/9567d59139a64afa82500c750e4406bb/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:10.951223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f3d10-327c-461a-9928-c2dc86359b4d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:10.951510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f3d10-327c-461a-9928-c2dc86359b4d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:10.951783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f3d10-327c-461a-9928-c2dc86359b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:10.978559 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a23f3d10-327c-461a-9928-c2dc86359b4d None None] MFA Rules not processed for user `9567d59139a64afa82500c750e4406bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:10.979737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a23f3d10-327c-461a-9928-c2dc86359b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:11.044494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1343/2689] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:11.056625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b415f89-30a7-4085-a7b3-75dcf29c3a65 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:11.058471 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9b415f89-30a7-4085-a7b3-75dcf29c3a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:11.121359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b415f89-30a7-4085-a7b3-75dcf29c3a65 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:11.125464 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b4961-ea16-4464-9f0c-c6283454dbec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:11.126295 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b4961-ea16-4464-9f0c-c6283454dbec service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:11.126567 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b4961-ea16-4464-9f0c-c6283454dbec service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:11.126838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b4961-ea16-4464-9f0c-c6283454dbec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:11.136140 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b4961-ea16-4464-9f0c-c6283454dbec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:11.137641 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b4961-ea16-4464-9f0c-c6283454dbec service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:11.147418 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1347/2690] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:10:11.167463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f3d10-327c-461a-9928-c2dc86359b4d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:11.231290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f3d10-327c-461a-9928-c2dc86359b4d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:11.236635 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:11.237514 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:11.237785 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:11.238055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] PATH_INFO: `/v3/users/9567d59139a64afa82500c750e4406bb` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:11.239266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9567d59139a64afa82500c750e4406bb)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:11.241246 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:11.290969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9567d59139a64afa82500c750e4406bb'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:11.291608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9567d59139a64afa82500c750e4406bb'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:11.295935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:11.299072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9567d59139a64afa82500c750e4406bb'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:11.300004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9567d59139a64afa82500c750e4406bb'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:11.312784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9567d59139a64afa82500c750e4406bb'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:11.327483 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1344/2691] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:10:11 2022] DELETE /identity/v3/users/9567d59139a64afa82500c750e4406bb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:11.332427 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304564e-7234-404f-aa9e-94a048b222f1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:11.332719 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304564e-7234-404f-aa9e-94a048b222f1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:11.332995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304564e-7234-404f-aa9e-94a048b222f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:11.351322 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6304564e-7234-404f-aa9e-94a048b222f1 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:11.352565 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6304564e-7234-404f-aa9e-94a048b222f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:11.424566 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1348/2692] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:11.434551 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:11.437147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:11.506037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f61091-0f8f-44bb-bb03-4f2f73e86522 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:11.510451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ba71b-4b89-4615-af29-aa6c67da6104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:11.511117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ba71b-4b89-4615-af29-aa6c67da6104 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:11.511385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ba71b-4b89-4615-af29-aa6c67da6104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:11.511656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ba71b-4b89-4615-af29-aa6c67da6104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:11.522093 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ba71b-4b89-4615-af29-aa6c67da6104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:11.523530 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ba71b-4b89-4615-af29-aa6c67da6104 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:11.534111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1345/2693] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:10:11.820064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304564e-7234-404f-aa9e-94a048b222f1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:11.821550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6304564e-7234-404f-aa9e-94a048b222f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:11.833955 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304564e-7234-404f-aa9e-94a048b222f1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:11.838649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:11.839533 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:11.839815 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:11.840235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] PATH_INFO: `/v3/projects/6b688b7b65aa4b6d8d2ce55ccd3a9279` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:11.841584 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b688b7b65aa4b6d8d2ce55ccd3a9279)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:11.843530 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:11.897908 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b688b7b65aa4b6d8d2ce55ccd3a9279', 'request_id': 'req-05a726c8-63f9-4223-8ee9-2e34b1af4af7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:11.904145 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b688b7b65aa4b6d8d2ce55ccd3a9279', 'request_id': 'req-05a726c8-63f9-4223-8ee9-2e34b1af4af7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:11.913858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b688b7b65aa4b6d8d2ce55ccd3a9279', 'request_id': 'req-05a726c8-63f9-4223-8ee9-2e34b1af4af7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:11.925779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] The token cache is being invalidate because project 6b688b7b65aa4b6d8d2ce55ccd3a9279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:10:11.926006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:11.926628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-05a726c8-63f9-4223-8ee9-2e34b1af4af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:11.928220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1349/2694] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:10:11 2022] DELETE /identity/v3/projects/6b688b7b65aa4b6d8d2ce55ccd3a9279 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:11.933259 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e70cba-f22a-4fc6-a02a-5b19920226dc None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:11.933543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e70cba-f22a-4fc6-a02a-5b19920226dc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:11.933817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e70cba-f22a-4fc6-a02a-5b19920226dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:11.951973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-97e70cba-f22a-4fc6-a02a-5b19920226dc None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:11.953359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-97e70cba-f22a-4fc6-a02a-5b19920226dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:12.026193 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1346/2695] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:12.092885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f192f-066d-4b63-8058-cbca0eaaa41f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:12.093164 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f192f-066d-4b63-8058-cbca0eaaa41f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:12.093534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f192f-066d-4b63-8058-cbca0eaaa41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:12.114293 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-423f192f-066d-4b63-8058-cbca0eaaa41f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:12.115497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-423f192f-066d-4b63-8058-cbca0eaaa41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:12.125743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1350/2696] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:12.131768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e70cba-f22a-4fc6-a02a-5b19920226dc None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:12.142973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e70cba-f22a-4fc6-a02a-5b19920226dc None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:12.149165 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308628b-7cd4-428d-877e-272b8e298fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:12.149833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308628b-7cd4-428d-877e-272b8e298fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:12.150107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308628b-7cd4-428d-877e-272b8e298fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:12.150384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308628b-7cd4-428d-877e-272b8e298fb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:12.151117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308628b-7cd4-428d-877e-272b8e298fb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:10:12.151418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308628b-7cd4-428d-877e-272b8e298fb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:12.153067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308628b-7cd4-428d-877e-272b8e298fb0 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:12.165846 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1347/2697] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:10:12.170304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f192f-066d-4b63-8058-cbca0eaaa41f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:12.181075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f192f-066d-4b63-8058-cbca0eaaa41f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:12.186396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183a6c3-b294-4c13-9bc0-c4905fbf0d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:12.187056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183a6c3-b294-4c13-9bc0-c4905fbf0d39 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:12.187334 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183a6c3-b294-4c13-9bc0-c4905fbf0d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:12.187685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183a6c3-b294-4c13-9bc0-c4905fbf0d39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:12.188545 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183a6c3-b294-4c13-9bc0-c4905fbf0d39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:12.190461 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183a6c3-b294-4c13-9bc0-c4905fbf0d39 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:12.205899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:12.222119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1351/2698] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:10:12 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:10:12.226475 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308628b-7cd4-428d-877e-272b8e298fb0 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:12.236689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308628b-7cd4-428d-877e-272b8e298fb0 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:12.302542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30512bb4-f46e-4bfe-a085-2a1ace750c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:12.303215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30512bb4-f46e-4bfe-a085-2a1ace750c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:12.303501 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30512bb4-f46e-4bfe-a085-2a1ace750c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:12.303776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30512bb4-f46e-4bfe-a085-2a1ace750c58 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:12.304513 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30512bb4-f46e-4bfe-a085-2a1ace750c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:12.306453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30512bb4-f46e-4bfe-a085-2a1ace750c58 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:12.314083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:12.331209 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1348/2699] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:10:12 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:10:12.335378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183a6c3-b294-4c13-9bc0-c4905fbf0d39 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:12.345976 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183a6c3-b294-4c13-9bc0-c4905fbf0d39 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:12.351147 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacc8f6-ee41-472c-8cfa-a82deaaedee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:12.351810 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacc8f6-ee41-472c-8cfa-a82deaaedee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:12.352091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacc8f6-ee41-472c-8cfa-a82deaaedee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:12.352389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacc8f6-ee41-472c-8cfa-a82deaaedee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:12.353115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacc8f6-ee41-472c-8cfa-a82deaaedee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:12.354806 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacc8f6-ee41-472c-8cfa-a82deaaedee2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:12.366207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1352/2700] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:10:12.370359 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30512bb4-f46e-4bfe-a085-2a1ace750c58 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:12.381422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30512bb4-f46e-4bfe-a085-2a1ace750c58 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:12.386944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d4c10-c6eb-496f-8600-1e217c8ed842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:12.387607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d4c10-c6eb-496f-8600-1e217c8ed842 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:12.387892 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d4c10-c6eb-496f-8600-1e217c8ed842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:12.388169 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d4c10-c6eb-496f-8600-1e217c8ed842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:12.389097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d4c10-c6eb-496f-8600-1e217c8ed842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:12.390708 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d4c10-c6eb-496f-8600-1e217c8ed842 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:12.402875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1349/2701] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:10:12.408258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacc8f6-ee41-472c-8cfa-a82deaaedee2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:12.425055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacc8f6-ee41-472c-8cfa-a82deaaedee2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:12.430675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e77270-3b7f-45e0-aed0-e954018afbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:12.431637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e77270-3b7f-45e0-aed0-e954018afbf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:12.431831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e77270-3b7f-45e0-aed0-e954018afbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:12.432142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e77270-3b7f-45e0-aed0-e954018afbf3 admin admin] PATH_INFO: `/v3/projects/ffc21aec264c4be8a8fc17c32981666c/users/f4e5f37d09f6420f953c5993dc80d86c/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:12.448381 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e77270-3b7f-45e0-aed0-e954018afbf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc21aec264c4be8a8fc17c32981666c, user_id=f4e5f37d09f6420f953c5993dc80d86c, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:12.451117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e77270-3b7f-45e0-aed0-e954018afbf3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:12.468058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1353/2702] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:10:12 2022] PUT /identity/v3/projects/ffc21aec264c4be8a8fc17c32981666c/users/f4e5f37d09f6420f953c5993dc80d86c/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:10:12.479154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee9bcc-3fb6-4893-8dcd-90802e071f55 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:12.479448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee9bcc-3fb6-4893-8dcd-90802e071f55 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:12.479728 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee9bcc-3fb6-4893-8dcd-90802e071f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:12.507604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a6ee9bcc-3fb6-4893-8dcd-90802e071f55 None None] MFA Rules not processed for user `f4e5f37d09f6420f953c5993dc80d86c`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:10:12.508851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a6ee9bcc-3fb6-4893-8dcd-90802e071f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:12.576790 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1350/2703] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:10:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:10:12.592151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e77270-3b7f-45e0-aed0-e954018afbf3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:12.594115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-02e77270-3b7f-45e0-aed0-e954018afbf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:12.682188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e77270-3b7f-45e0-aed0-e954018afbf3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:12.686208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ba5b5-4e2b-4021-a2bb-bf683bb4c71c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097b12fbf6494336b11ece5c498a4103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:12.686872 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ba5b5-4e2b-4021-a2bb-bf683bb4c71c service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:12.687161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ba5b5-4e2b-4021-a2bb-bf683bb4c71c service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:12.687444 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ba5b5-4e2b-4021-a2bb-bf683bb4c71c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:12.696280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ba5b5-4e2b-4021-a2bb-bf683bb4c71c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:12.697731 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ba5b5-4e2b-4021-a2bb-bf683bb4c71c service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:12.707155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1354/2704] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:10:22.573919 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:22.702574 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=2,keystone:SELECT=52 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:49.998292 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee9bcc-3fb6-4893-8dcd-90802e071f55 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:49.999851 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a6ee9bcc-3fb6-4893-8dcd-90802e071f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:50.058801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee9bcc-3fb6-4893-8dcd-90802e071f55 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:50.062671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc49de6-90d4-4cc5-8204-9ad61f3efb8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:50.063320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc49de6-90d4-4cc5-8204-9ad61f3efb8b service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:50.063602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc49de6-90d4-4cc5-8204-9ad61f3efb8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:50.063872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc49de6-90d4-4cc5-8204-9ad61f3efb8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:50.065769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6fc49de6-90d4-4cc5-8204-9ad61f3efb8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:50.142859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc49de6-90d4-4cc5-8204-9ad61f3efb8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:50.144354 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc49de6-90d4-4cc5-8204-9ad61f3efb8b service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:50.154799 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1351/2705] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:10:53.732076 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ba5b5-4e2b-4021-a2bb-bf683bb4c71c service glance] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:53.733526 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ca2ba5b5-4e2b-4021-a2bb-bf683bb4c71c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:53.813866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ba5b5-4e2b-4021-a2bb-bf683bb4c71c service glance] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:53.819543 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68dcb3-97fa-4ff9-aeaf-2b2ae3affb70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:53.820258 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68dcb3-97fa-4ff9-aeaf-2b2ae3affb70 service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:53.820583 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68dcb3-97fa-4ff9-aeaf-2b2ae3affb70 service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:53.820830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68dcb3-97fa-4ff9-aeaf-2b2ae3affb70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:53.822929 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fc68dcb3-97fa-4ff9-aeaf-2b2ae3affb70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:53.839657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68dcb3-97fa-4ff9-aeaf-2b2ae3affb70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:53.842047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68dcb3-97fa-4ff9-aeaf-2b2ae3affb70 service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:53.854849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1355/2706] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:10:56.837850 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc49de6-90d4-4cc5-8204-9ad61f3efb8b service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:56.839843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6fc49de6-90d4-4cc5-8204-9ad61f3efb8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:56.853177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc49de6-90d4-4cc5-8204-9ad61f3efb8b service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:56.857822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:56.858716 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:56.858994 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:56.859272 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] PATH_INFO: `/v3/users/5f8c2e5e6c1b4833b0e9af0de8814d13` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:56.873569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8c2e5e6c1b4833b0e9af0de8814d13)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:56.875510 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:56.926224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8c2e5e6c1b4833b0e9af0de8814d13'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:56.926944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8c2e5e6c1b4833b0e9af0de8814d13'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:56.934250 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8c2e5e6c1b4833b0e9af0de8814d13'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:56.934918 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8c2e5e6c1b4833b0e9af0de8814d13'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:56.946993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8c2e5e6c1b4833b0e9af0de8814d13'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:56.960467 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1352/2707] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:10:56 2022] DELETE /identity/v3/users/5f8c2e5e6c1b4833b0e9af0de8814d13 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:57.147451 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68dcb3-97fa-4ff9-aeaf-2b2ae3affb70 service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:57.158448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68dcb3-97fa-4ff9-aeaf-2b2ae3affb70 service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:57.218260 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:57.219139 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:57.219424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:57.219700 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] PATH_INFO: `/v3/projects/6ed25b82be2043b188d2682a5c01490a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:57.230359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed25b82be2043b188d2682a5c01490a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:57.232359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:57.280672 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed25b82be2043b188d2682a5c01490a', 'request_id': 'req-73563d24-4f44-457f-aab7-79b857db3e94'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.287181 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed25b82be2043b188d2682a5c01490a', 'request_id': 'req-73563d24-4f44-457f-aab7-79b857db3e94'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.296608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed25b82be2043b188d2682a5c01490a', 'request_id': 'req-73563d24-4f44-457f-aab7-79b857db3e94'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.309381 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] The token cache is being invalidate because project 6ed25b82be2043b188d2682a5c01490a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:10:57.309666 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.310312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.311891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1356/2708] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:10:57 2022] DELETE /identity/v3/projects/6ed25b82be2043b188d2682a5c01490a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:57.316633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:57.318081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:57.332228 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3973e1c-1ef1-4d61-a33f-66c7c1a926c7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:57.348752 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:57.409252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:57.410177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:57.410458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:57.410720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] PATH_INFO: `/v3/users/8bd59999ed69444aaeb3fe60c4befcca` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:57.426971 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bd59999ed69444aaeb3fe60c4befcca)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:57.429045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:57.477368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bd59999ed69444aaeb3fe60c4befcca'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.478003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bd59999ed69444aaeb3fe60c4befcca'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.485287 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bd59999ed69444aaeb3fe60c4befcca'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.486003 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bd59999ed69444aaeb3fe60c4befcca'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.499798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bd59999ed69444aaeb3fe60c4befcca'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.513986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1353/2709] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:10:57 2022] DELETE /identity/v3/users/8bd59999ed69444aaeb3fe60c4befcca => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:57.718195 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:57.720202 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:57.740623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73563d24-4f44-457f-aab7-79b857db3e94 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:57.762822 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:57.820605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:57.821469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:57.821856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:57.822193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] PATH_INFO: `/v3/projects/ba0054f6e68e46749271aa7af502c931` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:57.838004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0054f6e68e46749271aa7af502c931)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:57.840021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:57.888739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0054f6e68e46749271aa7af502c931', 'request_id': 'req-8b0b6313-66df-40f3-b590-3606c7cf104e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.895286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0054f6e68e46749271aa7af502c931', 'request_id': 'req-8b0b6313-66df-40f3-b590-3606c7cf104e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.904342 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0054f6e68e46749271aa7af502c931', 'request_id': 'req-8b0b6313-66df-40f3-b590-3606c7cf104e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.915386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] The token cache is being invalidate because project ba0054f6e68e46749271aa7af502c931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:10:57.915668 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.916319 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:10:57.917868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1357/2710] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:10:57 2022] DELETE /identity/v3/projects/ba0054f6e68e46749271aa7af502c931 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:10:59.350435 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:10:59.351868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:59.431658 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3300f8-4d8a-4f52-81cf-3e4d75b47cdd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:10:59.446334 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:10:59.504237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cdd3c-0751-4dea-ad8d-7ce9f5a3bd72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:10:59.504897 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cdd3c-0751-4dea-ad8d-7ce9f5a3bd72 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:10:59.505177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cdd3c-0751-4dea-ad8d-7ce9f5a3bd72 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:10:59.505449 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cdd3c-0751-4dea-ad8d-7ce9f5a3bd72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:10:59.507515 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1e4cdd3c-0751-4dea-ad8d-7ce9f5a3bd72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:10:59.565878 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cdd3c-0751-4dea-ad8d-7ce9f5a3bd72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:10:59.567486 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cdd3c-0751-4dea-ad8d-7ce9f5a3bd72 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:10:59.577091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1354/2711] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:10:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:11:01.129739 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:01.131371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:01.198533 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b6313-66df-40f3-b590-3606c7cf104e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:01.203532 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:01.204612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:01.204894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:01.205208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] PATH_INFO: `/v3/users/f4e5f37d09f6420f953c5993dc80d86c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:01.206434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e5f37d09f6420f953c5993dc80d86c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:01.208402 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:01.249771 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:01.260186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e5f37d09f6420f953c5993dc80d86c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:01.270378 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e5f37d09f6420f953c5993dc80d86c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:01.271127 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e5f37d09f6420f953c5993dc80d86c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:01.280984 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e5f37d09f6420f953c5993dc80d86c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:01.294410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e5f37d09f6420f953c5993dc80d86c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:01.300991 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1358/2712] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:11:01 2022] DELETE /identity/v3/users/f4e5f37d09f6420f953c5993dc80d86c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:11:01.305798 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbb468-90e8-4e8e-9167-7f7422970043 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:01.306275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbb468-90e8-4e8e-9167-7f7422970043 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:01.306275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbb468-90e8-4e8e-9167-7f7422970043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:01.326162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d4dbb468-90e8-4e8e-9167-7f7422970043 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:11:01.327404 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d4dbb468-90e8-4e8e-9167-7f7422970043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:01.389844 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1355/2713] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:11:01.400244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:01.401776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:01.461148 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301dc6e-6cc6-4b09-a153-16fb622b7b78 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:01.465233 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b283ed6-ae3c-49ea-9775-d30a7719c942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:01.465882 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b283ed6-ae3c-49ea-9775-d30a7719c942 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:01.466160 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b283ed6-ae3c-49ea-9775-d30a7719c942 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:01.466430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b283ed6-ae3c-49ea-9775-d30a7719c942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:01.475916 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b283ed6-ae3c-49ea-9775-d30a7719c942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:01.477386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b283ed6-ae3c-49ea-9775-d30a7719c942 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:01.487521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1359/2714] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:11:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:11:01.772672 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbb468-90e8-4e8e-9167-7f7422970043 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:01.774343 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d4dbb468-90e8-4e8e-9167-7f7422970043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:01.787427 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbb468-90e8-4e8e-9167-7f7422970043 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:01.793069 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:01.794026 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:01.795262 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:01.795543 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] PATH_INFO: `/v3/projects/ffc21aec264c4be8a8fc17c32981666c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:01.797603 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc21aec264c4be8a8fc17c32981666c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:01.799815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:01.872535 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc21aec264c4be8a8fc17c32981666c', 'request_id': 'req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:01.873215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:01.879280 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc21aec264c4be8a8fc17c32981666c', 'request_id': 'req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:01.890862 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc21aec264c4be8a8fc17c32981666c', 'request_id': 'req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:01.901611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] The token cache is being invalidate because project ffc21aec264c4be8a8fc17c32981666c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:11:01.901898 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:01.902531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0cd08cd3-9193-4d78-853b-d7d3a95f8c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:01.904102 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1356/2715] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:11:01 2022] DELETE /identity/v3/projects/ffc21aec264c4be8a8fc17c32981666c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:11:01.909243 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee507b-7f6f-49f5-a05c-291051cb51e4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:01.909534 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee507b-7f6f-49f5-a05c-291051cb51e4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:01.909807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee507b-7f6f-49f5-a05c-291051cb51e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:01.928946 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-69ee507b-7f6f-49f5-a05c-291051cb51e4 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:11:01.930252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-69ee507b-7f6f-49f5-a05c-291051cb51e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:01.988701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1360/2716] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:11:01.997596 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91f6b2-f472-4560-b1df-3db19299e133 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:01.997896 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91f6b2-f472-4560-b1df-3db19299e133 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:01.998134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91f6b2-f472-4560-b1df-3db19299e133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:02.016766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-de91f6b2-f472-4560-b1df-3db19299e133 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:11:02.017976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-de91f6b2-f472-4560-b1df-3db19299e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:02.026108 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1357/2717] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:11:02.030599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee507b-7f6f-49f5-a05c-291051cb51e4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:02.040356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee507b-7f6f-49f5-a05c-291051cb51e4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:02.044954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702bea1-abb3-43ec-a5a4-abffa54f68e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:02.045606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702bea1-abb3-43ec-a5a4-abffa54f68e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:02.045884 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702bea1-abb3-43ec-a5a4-abffa54f68e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:02.046154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702bea1-abb3-43ec-a5a4-abffa54f68e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:02.047386 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702bea1-abb3-43ec-a5a4-abffa54f68e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:11:02.047682 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702bea1-abb3-43ec-a5a4-abffa54f68e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:02.049340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702bea1-abb3-43ec-a5a4-abffa54f68e2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:02.061424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1361/2718] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:11:02.065915 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91f6b2-f472-4560-b1df-3db19299e133 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:02.075945 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91f6b2-f472-4560-b1df-3db19299e133 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:02.080767 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fa49b-ba1f-4095-8bb7-3ad1deb9e648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:02.081432 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fa49b-ba1f-4095-8bb7-3ad1deb9e648 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:02.081710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fa49b-ba1f-4095-8bb7-3ad1deb9e648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:02.081976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fa49b-ba1f-4095-8bb7-3ad1deb9e648 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:02.082671 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fa49b-ba1f-4095-8bb7-3ad1deb9e648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:02.084611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fa49b-ba1f-4095-8bb7-3ad1deb9e648 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:02.116802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1358/2719] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:11:02 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:11:02.118259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702bea1-abb3-43ec-a5a4-abffa54f68e2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:02.128932 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702bea1-abb3-43ec-a5a4-abffa54f68e2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:02.172593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:02.191707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c680b6-a019-46df-ba91-d02164a11a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:02.192653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c680b6-a019-46df-ba91-d02164a11a97 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:02.193059 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c680b6-a019-46df-ba91-d02164a11a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:02.193203 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c680b6-a019-46df-ba91-d02164a11a97 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:02.193972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c680b6-a019-46df-ba91-d02164a11a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:02.196231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c680b6-a019-46df-ba91-d02164a11a97 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:02.219459 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1362/2720] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:11:02 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:11:02.224755 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fa49b-ba1f-4095-8bb7-3ad1deb9e648 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:02.236937 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fa49b-ba1f-4095-8bb7-3ad1deb9e648 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:02.243688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f9455-89a9-4e56-88b1-083f8270918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:02.244695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f9455-89a9-4e56-88b1-083f8270918f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:02.245099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f9455-89a9-4e56-88b1-083f8270918f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:02.245586 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f9455-89a9-4e56-88b1-083f8270918f admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:02.246487 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f9455-89a9-4e56-88b1-083f8270918f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:02.248213 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f9455-89a9-4e56-88b1-083f8270918f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:02.267267 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1359/2721] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:11:02.268663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c680b6-a019-46df-ba91-d02164a11a97 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:02.280225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c680b6-a019-46df-ba91-d02164a11a97 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:02.286089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789c152-62b6-4246-b691-014405caea9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:02.286776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789c152-62b6-4246-b691-014405caea9e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:02.287058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789c152-62b6-4246-b691-014405caea9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:02.287328 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789c152-62b6-4246-b691-014405caea9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:02.288208 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789c152-62b6-4246-b691-014405caea9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:02.289838 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789c152-62b6-4246-b691-014405caea9e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:02.304151 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1363/2722] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:11:02.305294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f9455-89a9-4e56-88b1-083f8270918f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:02.315608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f9455-89a9-4e56-88b1-083f8270918f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:02.321179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99200b-6fdd-44e3-b26f-30c2878a1b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:02.322782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99200b-6fdd-44e3-b26f-30c2878a1b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:02.323052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99200b-6fdd-44e3-b26f-30c2878a1b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:02.323329 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99200b-6fdd-44e3-b26f-30c2878a1b9f admin admin] PATH_INFO: `/v3/projects/12cbfddd45eb4af9a2581568034655be/users/51af135fb39a4fcb896d2c995661c6c8/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:02.355111 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99200b-6fdd-44e3-b26f-30c2878a1b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=12cbfddd45eb4af9a2581568034655be, user_id=51af135fb39a4fcb896d2c995661c6c8, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:02.357815 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99200b-6fdd-44e3-b26f-30c2878a1b9f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:02.372172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1360/2723] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:11:02 2022] PUT /identity/v3/projects/12cbfddd45eb4af9a2581568034655be/users/51af135fb39a4fcb896d2c995661c6c8/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:11:02.383498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df5813-76ed-48db-84b3-f8d6ed88f027 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:02.383681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df5813-76ed-48db-84b3-f8d6ed88f027 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:02.383935 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df5813-76ed-48db-84b3-f8d6ed88f027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:02.410121 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e4df5813-76ed-48db-84b3-f8d6ed88f027 None None] MFA Rules not processed for user `51af135fb39a4fcb896d2c995661c6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:11:02.411321 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e4df5813-76ed-48db-84b3-f8d6ed88f027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:02.478061 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1364/2724] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:11:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:11:02.489319 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99200b-6fdd-44e3-b26f-30c2878a1b9f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:02.491302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8b99200b-6fdd-44e3-b26f-30c2878a1b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:02.561274 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99200b-6fdd-44e3-b26f-30c2878a1b9f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:02.565579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57712234-8d82-47cb-83ca-1fdc294093a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:02.566335 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57712234-8d82-47cb-83ca-1fdc294093a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:02.566602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57712234-8d82-47cb-83ca-1fdc294093a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:02.566894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57712234-8d82-47cb-83ca-1fdc294093a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:02.577982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57712234-8d82-47cb-83ca-1fdc294093a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:02.579463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57712234-8d82-47cb-83ca-1fdc294093a4 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:02.603905 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1361/2725] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:11:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:11:05.219278 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df5813-76ed-48db-84b3-f8d6ed88f027 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:05.292978 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df5813-76ed-48db-84b3-f8d6ed88f027 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:05.298472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:05.299366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:05.299651 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:05.299925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] PATH_INFO: `/v3/users/51af135fb39a4fcb896d2c995661c6c8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:05.301210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51af135fb39a4fcb896d2c995661c6c8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:05.303201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:05.352428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51af135fb39a4fcb896d2c995661c6c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:05.357231 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:05.360445 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51af135fb39a4fcb896d2c995661c6c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:05.361463 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51af135fb39a4fcb896d2c995661c6c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:05.370102 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51af135fb39a4fcb896d2c995661c6c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:05.383298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51af135fb39a4fcb896d2c995661c6c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:05.393077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1365/2726] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:11:05 2022] DELETE /identity/v3/users/51af135fb39a4fcb896d2c995661c6c8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:11:05.397683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58267d-f409-4068-9ed9-0a7139ddfe0a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:05.398192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58267d-f409-4068-9ed9-0a7139ddfe0a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:05.398192 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58267d-f409-4068-9ed9-0a7139ddfe0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:05.416175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ac58267d-f409-4068-9ed9-0a7139ddfe0a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:11:05.417763 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ac58267d-f409-4068-9ed9-0a7139ddfe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:05.482787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1362/2727] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:11:05.492943 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:05.494497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:05.555280 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b351a8-261b-475d-adaa-67e2f3ea7773 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:05.559308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a72ce-7c41-4ccd-b9d2-cfd56a125f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:05.559972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a72ce-7c41-4ccd-b9d2-cfd56a125f33 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:05.560250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a72ce-7c41-4ccd-b9d2-cfd56a125f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:05.560527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a72ce-7c41-4ccd-b9d2-cfd56a125f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:05.573005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a72ce-7c41-4ccd-b9d2-cfd56a125f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:05.574529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a72ce-7c41-4ccd-b9d2-cfd56a125f33 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:05.584907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1366/2728] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:11:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:11:05.940944 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58267d-f409-4068-9ed9-0a7139ddfe0a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:05.942656 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ac58267d-f409-4068-9ed9-0a7139ddfe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:05.956268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58267d-f409-4068-9ed9-0a7139ddfe0a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:05.961156 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:05.962025 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:05.962304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:05.962574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] PATH_INFO: `/v3/projects/12cbfddd45eb4af9a2581568034655be` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:05.963763 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12cbfddd45eb4af9a2581568034655be)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:05.965726 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:05.977719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:06.016450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12cbfddd45eb4af9a2581568034655be', 'request_id': 'req-778d9e1d-777e-461d-9d67-224f37a0cf91'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:06.023289 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12cbfddd45eb4af9a2581568034655be', 'request_id': 'req-778d9e1d-777e-461d-9d67-224f37a0cf91'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:06.032883 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12cbfddd45eb4af9a2581568034655be', 'request_id': 'req-778d9e1d-777e-461d-9d67-224f37a0cf91'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:06.045162 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] The token cache is being invalidate because project 12cbfddd45eb4af9a2581568034655be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:11:06.045453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:06.046099 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-778d9e1d-777e-461d-9d67-224f37a0cf91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:06.048083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1363/2729] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:11:05 2022] DELETE /identity/v3/projects/12cbfddd45eb4af9a2581568034655be => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:11:06.054077 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ee115-dbff-4c8a-87df-050494dda7cb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:06.054430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ee115-dbff-4c8a-87df-050494dda7cb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:06.054861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ee115-dbff-4c8a-87df-050494dda7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:06.079460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-538ee115-dbff-4c8a-87df-050494dda7cb None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:11:06.081351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-538ee115-dbff-4c8a-87df-050494dda7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:06.151787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1367/2730] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:11:06.159959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e55f9-43dc-43bd-b68c-f62483dce3c5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:06.160296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e55f9-43dc-43bd-b68c-f62483dce3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:06.160579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e55f9-43dc-43bd-b68c-f62483dce3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:06.182315 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d22e55f9-43dc-43bd-b68c-f62483dce3c5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:11:06.183576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d22e55f9-43dc-43bd-b68c-f62483dce3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:06.193327 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1364/2731] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:11:06.198579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ee115-dbff-4c8a-87df-050494dda7cb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:06.209272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ee115-dbff-4c8a-87df-050494dda7cb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:06.215890 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb9279-a39d-4e22-b723-9270dcb27a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:06.216971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb9279-a39d-4e22-b723-9270dcb27a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:06.217438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb9279-a39d-4e22-b723-9270dcb27a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:06.217891 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb9279-a39d-4e22-b723-9270dcb27a12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:06.218628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb9279-a39d-4e22-b723-9270dcb27a12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:11:06.218933 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb9279-a39d-4e22-b723-9270dcb27a12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:06.220656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb9279-a39d-4e22-b723-9270dcb27a12 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:06.237196 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1368/2732] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:11:06.242636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e55f9-43dc-43bd-b68c-f62483dce3c5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:06.252760 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e55f9-43dc-43bd-b68c-f62483dce3c5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:06.259117 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1e125-3950-4d80-a0f8-bd31973337a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:06.259712 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1e125-3950-4d80-a0f8-bd31973337a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:06.259993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1e125-3950-4d80-a0f8-bd31973337a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:06.260306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1e125-3950-4d80-a0f8-bd31973337a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:06.261011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1e125-3950-4d80-a0f8-bd31973337a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:06.262938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1e125-3950-4d80-a0f8-bd31973337a3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:06.291990 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1365/2733] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:11:06 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:11:06.296662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb9279-a39d-4e22-b723-9270dcb27a12 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:06.307748 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb9279-a39d-4e22-b723-9270dcb27a12 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:06.367537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4e692-5d83-411c-bd08-7a55af07d7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:06.368255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e692-5d83-411c-bd08-7a55af07d7ac admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:06.368542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e692-5d83-411c-bd08-7a55af07d7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:06.368924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e692-5d83-411c-bd08-7a55af07d7ac admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:06.369634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4e692-5d83-411c-bd08-7a55af07d7ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:06.371628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4e692-5d83-411c-bd08-7a55af07d7ac admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:06.378236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:06.395789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1369/2734] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:11:06 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:11:06.400476 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1e125-3950-4d80-a0f8-bd31973337a3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:06.411478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1e125-3950-4d80-a0f8-bd31973337a3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:06.417451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e3290-a1ff-44ce-8d14-e153a6b597fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:06.418137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3290-a1ff-44ce-8d14-e153a6b597fc admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:06.418416 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3290-a1ff-44ce-8d14-e153a6b597fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:06.418684 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3290-a1ff-44ce-8d14-e153a6b597fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:06.419417 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e3290-a1ff-44ce-8d14-e153a6b597fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:06.421058 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e3290-a1ff-44ce-8d14-e153a6b597fc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:06.432412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1366/2735] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:11:06.437776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4e692-5d83-411c-bd08-7a55af07d7ac admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:06.448144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4e692-5d83-411c-bd08-7a55af07d7ac admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:06.453924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2599c92-fa66-4df0-bafa-2b3399d0aeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:06.454590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2599c92-fa66-4df0-bafa-2b3399d0aeda admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:06.454868 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2599c92-fa66-4df0-bafa-2b3399d0aeda admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:06.455141 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2599c92-fa66-4df0-bafa-2b3399d0aeda admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:06.455870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2599c92-fa66-4df0-bafa-2b3399d0aeda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:06.457509 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2599c92-fa66-4df0-bafa-2b3399d0aeda admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:06.468330 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1370/2736] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:11:06.472615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e3290-a1ff-44ce-8d14-e153a6b597fc admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:06.482762 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e3290-a1ff-44ce-8d14-e153a6b597fc admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:06.487758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199031d-70a2-42a3-a6e7-219b3af55c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:06.488681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199031d-70a2-42a3-a6e7-219b3af55c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:06.488959 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199031d-70a2-42a3-a6e7-219b3af55c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:06.489214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199031d-70a2-42a3-a6e7-219b3af55c14 admin admin] PATH_INFO: `/v3/projects/fb6ab49c539b421ab17675534e7e695d/users/1952b09ab77d4e0d9fe3a31448790859/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:06.505567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199031d-70a2-42a3-a6e7-219b3af55c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6ab49c539b421ab17675534e7e695d, user_id=1952b09ab77d4e0d9fe3a31448790859, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:06.508219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199031d-70a2-42a3-a6e7-219b3af55c14 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:06.522831 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1367/2737] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:11:06 2022] PUT /identity/v3/projects/fb6ab49c539b421ab17675534e7e695d/users/1952b09ab77d4e0d9fe3a31448790859/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:11:06.534827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30090bc4-fd3a-4997-a1ae-1480b67ac050 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:06.535153 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30090bc4-fd3a-4997-a1ae-1480b67ac050 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:06.535531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30090bc4-fd3a-4997-a1ae-1480b67ac050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:06.566834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-30090bc4-fd3a-4997-a1ae-1480b67ac050 None None] MFA Rules not processed for user `1952b09ab77d4e0d9fe3a31448790859`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:11:06.568051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-30090bc4-fd3a-4997-a1ae-1480b67ac050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:06.634772 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1371/2738] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:11:06.646331 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199031d-70a2-42a3-a6e7-219b3af55c14 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:06.648326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2199031d-70a2-42a3-a6e7-219b3af55c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:06.712237 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199031d-70a2-42a3-a6e7-219b3af55c14 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:06.716769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd1053-a8d9-4eb1-a6af-4fd54fe8840b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:06.717467 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd1053-a8d9-4eb1-a6af-4fd54fe8840b service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:06.717750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd1053-a8d9-4eb1-a6af-4fd54fe8840b service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:06.718023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd1053-a8d9-4eb1-a6af-4fd54fe8840b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:06.727114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd1053-a8d9-4eb1-a6af-4fd54fe8840b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:06.728663 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd1053-a8d9-4eb1-a6af-4fd54fe8840b service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:06.737803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1368/2739] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:11:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:11:10.452013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373471e1-6b59-4e02-95ba-13a856fea68e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:10.452576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373471e1-6b59-4e02-95ba-13a856fea68e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:10.452696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373471e1-6b59-4e02-95ba-13a856fea68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:10.470662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-373471e1-6b59-4e02-95ba-13a856fea68e None None] MFA Rules not processed for user `68f6997db7c141529c37cc64955df779`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:11:10.471870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-373471e1-6b59-4e02-95ba-13a856fea68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:10.538632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1372/2740] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:11:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:11:10.865293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd1053-a8d9-4eb1-a6af-4fd54fe8840b service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:10.867050 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fedd1053-a8d9-4eb1-a6af-4fd54fe8840b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:10.915137 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:10.936434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd1053-a8d9-4eb1-a6af-4fd54fe8840b service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:10.940787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123d2cd-f60f-4a54-951e-27acf26b1fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:10.941451 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123d2cd-f60f-4a54-951e-27acf26b1fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:10.941733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123d2cd-f60f-4a54-951e-27acf26b1fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:10.942006 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123d2cd-f60f-4a54-951e-27acf26b1fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:10.943924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3123d2cd-f60f-4a54-951e-27acf26b1fa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:10.953989 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123d2cd-f60f-4a54-951e-27acf26b1fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:10.955425 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123d2cd-f60f-4a54-951e-27acf26b1fa5 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:10.963570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1369/2741] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:11:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:11:20.530901 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=3,keystone:SELECT=64 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:20.959832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=8 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:27.706698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373471e1-6b59-4e02-95ba-13a856fea68e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:27.708311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-373471e1-6b59-4e02-95ba-13a856fea68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:27.791809 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373471e1-6b59-4e02-95ba-13a856fea68e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:27.797634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdee7b8-472f-4be0-bca7-af2679aeb226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:27.798312 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee7b8-472f-4be0-bca7-af2679aeb226 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:27.798590 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee7b8-472f-4be0-bca7-af2679aeb226 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:27.798917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee7b8-472f-4be0-bca7-af2679aeb226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:27.800775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3cdee7b8-472f-4be0-bca7-af2679aeb226 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:27.814191 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdee7b8-472f-4be0-bca7-af2679aeb226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:27.815834 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdee7b8-472f-4be0-bca7-af2679aeb226 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:27.827828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1373/2742] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:11:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:11:34.826065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123d2cd-f60f-4a54-951e-27acf26b1fa5 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:34.827857 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3123d2cd-f60f-4a54-951e-27acf26b1fa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:34.905110 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123d2cd-f60f-4a54-951e-27acf26b1fa5 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:34.911779 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288ccb3-82ea-4005-84eb-a895d1c6f147 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:34.912494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288ccb3-82ea-4005-84eb-a895d1c6f147 service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:34.912775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288ccb3-82ea-4005-84eb-a895d1c6f147 service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:34.913046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288ccb3-82ea-4005-84eb-a895d1c6f147 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:34.915092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1288ccb3-82ea-4005-84eb-a895d1c6f147 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:34.928939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288ccb3-82ea-4005-84eb-a895d1c6f147 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:34.930539 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288ccb3-82ea-4005-84eb-a895d1c6f147 service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:34.941158 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1370/2743] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:11:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:11:37.822915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=34 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:44.941914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=32 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:47.371600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdee7b8-472f-4be0-bca7-af2679aeb226 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:47.439110 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdee7b8-472f-4be0-bca7-af2679aeb226 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:47.444327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:47.445215 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:47.445497 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:47.445773 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] PATH_INFO: `/v3/users/1952b09ab77d4e0d9fe3a31448790859` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:47.447152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1952b09ab77d4e0d9fe3a31448790859)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:47.449154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:47.501198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1952b09ab77d4e0d9fe3a31448790859'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:47.508273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1952b09ab77d4e0d9fe3a31448790859'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:47.508973 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1952b09ab77d4e0d9fe3a31448790859'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:47.518366 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1952b09ab77d4e0d9fe3a31448790859'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:47.535839 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1952b09ab77d4e0d9fe3a31448790859'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:47.544897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1374/2744] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:11:47 2022] DELETE /identity/v3/users/1952b09ab77d4e0d9fe3a31448790859 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:11:47.550482 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e51f8-9c2f-4548-9221-e047f46e4f2e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:47.550995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e51f8-9c2f-4548-9221-e047f46e4f2e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:47.550995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e51f8-9c2f-4548-9221-e047f46e4f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:47.568526 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-af1e51f8-9c2f-4548-9221-e047f46e4f2e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:11:47.569725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-af1e51f8-9c2f-4548-9221-e047f46e4f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:47.628212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1371/2745] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:11:47.638775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:47.640563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:47.702957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2180a34-4c8a-40df-83ff-ea8362329b29 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:47.707062 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8c8c7-3c2a-4251-9fbe-5b2b0d3098b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:47.707710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8c8c7-3c2a-4251-9fbe-5b2b0d3098b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:47.707990 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8c8c7-3c2a-4251-9fbe-5b2b0d3098b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:47.708308 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8c8c7-3c2a-4251-9fbe-5b2b0d3098b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:47.718126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8c8c7-3c2a-4251-9fbe-5b2b0d3098b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:47.719559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8c8c7-3c2a-4251-9fbe-5b2b0d3098b1 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:47.731836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1375/2746] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:11:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:11:48.006731 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e51f8-9c2f-4548-9221-e047f46e4f2e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:11:48.008236 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-af1e51f8-9c2f-4548-9221-e047f46e4f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:11:48.021518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e51f8-9c2f-4548-9221-e047f46e4f2e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:11:48.026940 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:11:48.027950 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:11:48.028269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:11:48.028545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] PATH_INFO: `/v3/projects/fb6ab49c539b421ab17675534e7e695d` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:11:48.029858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6ab49c539b421ab17675534e7e695d)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:11:48.031792 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:11:48.084133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6ab49c539b421ab17675534e7e695d', 'request_id': 'req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:48.090636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6ab49c539b421ab17675534e7e695d', 'request_id': 'req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:48.099761 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6ab49c539b421ab17675534e7e695d', 'request_id': 'req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:48.110769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] The token cache is being invalidate because project fb6ab49c539b421ab17675534e7e695d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:11:48.111051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:48.111660 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:11:48.113219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1372/2747] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:11:48 2022] DELETE /identity/v3/projects/fb6ab49c539b421ab17675534e7e695d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:11:57.725510 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:11:58.107563 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:12:20.554186 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8c8c7-3c2a-4251-9fbe-5b2b0d3098b1 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:12:20.563351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8c8c7-3c2a-4251-9fbe-5b2b0d3098b1 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:12:20.621608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f33c46-e0ab-4744-a9d6-9fc0870e06cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:12:20.622569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f33c46-e0ab-4744-a9d6-9fc0870e06cc service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:12:20.622964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f33c46-e0ab-4744-a9d6-9fc0870e06cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:12:20.623359 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f33c46-e0ab-4744-a9d6-9fc0870e06cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:12:20.626156 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-15f33c46-e0ab-4744-a9d6-9fc0870e06cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:12:20.705574 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f33c46-e0ab-4744-a9d6-9fc0870e06cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:12:20.707038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f33c46-e0ab-4744-a9d6-9fc0870e06cc service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:12:20.721080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1376/2748] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:12:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:12:21.000793 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:12:21.002422 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:12:21.014972 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25ffd4-7603-46ea-821b-bd1fe18b93ee admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:12:21.018670 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840662c-3bb8-4c7c-90b6-6c1943563fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:12:21.019504 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840662c-3bb8-4c7c-90b6-6c1943563fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:12:21.019784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840662c-3bb8-4c7c-90b6-6c1943563fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:12:21.020194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840662c-3bb8-4c7c-90b6-6c1943563fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:12:21.022017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5840662c-3bb8-4c7c-90b6-6c1943563fa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:12:21.032210 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840662c-3bb8-4c7c-90b6-6c1943563fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:12:21.033696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840662c-3bb8-4c7c-90b6-6c1943563fa5 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:12:21.042587 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1373/2749] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:12:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:12:21.146606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f33c46-e0ab-4744-a9d6-9fc0870e06cc service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:12:21.148425 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-15f33c46-e0ab-4744-a9d6-9fc0870e06cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:12:21.160941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f33c46-e0ab-4744-a9d6-9fc0870e06cc service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:12:21.165303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf6583-07c7-4a16-ab94-d37edadb55e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:12:21.165947 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf6583-07c7-4a16-ab94-d37edadb55e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:12:21.166269 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf6583-07c7-4a16-ab94-d37edadb55e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:12:21.166553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf6583-07c7-4a16-ab94-d37edadb55e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:12:21.168524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aaaf6583-07c7-4a16-ab94-d37edadb55e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:12:21.183119 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf6583-07c7-4a16-ab94-d37edadb55e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:12:21.184600 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf6583-07c7-4a16-ab94-d37edadb55e6 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:12:21.195374 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1377/2750] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:12:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:12:22.224860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840662c-3bb8-4c7c-90b6-6c1943563fa5 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:12:22.235936 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840662c-3bb8-4c7c-90b6-6c1943563fa5 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:12:22.240351 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54e7ce-b5f6-4bf3-ba09-d0e019eddd93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:12:22.241641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54e7ce-b5f6-4bf3-ba09-d0e019eddd93 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:12:22.241956 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54e7ce-b5f6-4bf3-ba09-d0e019eddd93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:12:22.242234 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54e7ce-b5f6-4bf3-ba09-d0e019eddd93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:12:22.244220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9a54e7ce-b5f6-4bf3-ba09-d0e019eddd93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:12:22.259601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54e7ce-b5f6-4bf3-ba09-d0e019eddd93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:12:22.261419 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54e7ce-b5f6-4bf3-ba09-d0e019eddd93 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:12:22.274284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1374/2751] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:12:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:12:31.190801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=60 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:12:32.269045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=12 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:13:00.547789 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87795f72-12e5-4aa0-a6bb-bc166a9b0aaf None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:13:00.548344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87795f72-12e5-4aa0-a6bb-bc166a9b0aaf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:13:00.548458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87795f72-12e5-4aa0-a6bb-bc166a9b0aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:13:00.567400 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-87795f72-12e5-4aa0-a6bb-bc166a9b0aaf None None] MFA Rules not processed for user `68f6997db7c141529c37cc64955df779`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:13:00.568957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-87795f72-12e5-4aa0-a6bb-bc166a9b0aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:13:00.629707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1378/2752] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:13:00.971754 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54e7ce-b5f6-4bf3-ba09-d0e019eddd93 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:13:00.985067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54e7ce-b5f6-4bf3-ba09-d0e019eddd93 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:13:00.990387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69c378-cefc-4a30-83c4-03ebfb541834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:13:00.991063 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69c378-cefc-4a30-83c4-03ebfb541834 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:13:00.991342 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69c378-cefc-4a30-83c4-03ebfb541834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:13:00.991609 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69c378-cefc-4a30-83c4-03ebfb541834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:13:00.993962 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-be69c378-cefc-4a30-83c4-03ebfb541834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:13:01.005249 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69c378-cefc-4a30-83c4-03ebfb541834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:13:01.006686 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69c378-cefc-4a30-83c4-03ebfb541834 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:13:01.014920 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1375/2753] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:13:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:13:10.618241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=25 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:13:11.011362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=6 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:14:07.997347 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87795f72-12e5-4aa0-a6bb-bc166a9b0aaf None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:08.008994 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87795f72-12e5-4aa0-a6bb-bc166a9b0aaf None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:08.013360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2117776-9dd1-4444-a3f3-2b319eda26fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:08.014007 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117776-9dd1-4444-a3f3-2b319eda26fc service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:08.014282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117776-9dd1-4444-a3f3-2b319eda26fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:08.014550 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117776-9dd1-4444-a3f3-2b319eda26fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:08.016742 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c2117776-9dd1-4444-a3f3-2b319eda26fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:08.085006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2117776-9dd1-4444-a3f3-2b319eda26fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:08.086444 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2117776-9dd1-4444-a3f3-2b319eda26fc service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:08.095557 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1379/2754] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:14:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:14:09.081706 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69c378-cefc-4a30-83c4-03ebfb541834 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:09.094681 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69c378-cefc-4a30-83c4-03ebfb541834 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:09.101333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992ff44-9392-460c-ba8a-ae971c1eb6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:09.102390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992ff44-9392-460c-ba8a-ae971c1eb6da service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:09.102759 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992ff44-9392-460c-ba8a-ae971c1eb6da service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:09.103128 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992ff44-9392-460c-ba8a-ae971c1eb6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:09.105364 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6992ff44-9392-460c-ba8a-ae971c1eb6da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:09.116986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992ff44-9392-460c-ba8a-ae971c1eb6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:09.118843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992ff44-9392-460c-ba8a-ae971c1eb6da service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:09.130125 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1376/2755] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:14:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:14:11.533780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2117776-9dd1-4444-a3f3-2b319eda26fc service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:11.535574 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c2117776-9dd1-4444-a3f3-2b319eda26fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:11.555004 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2117776-9dd1-4444-a3f3-2b319eda26fc service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:11.564926 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69547dbc-f198-465f-8b76-aa7d0bdc4aa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:11.565849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69547dbc-f198-465f-8b76-aa7d0bdc4aa9 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:11.566796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69547dbc-f198-465f-8b76-aa7d0bdc4aa9 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:11.567080 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69547dbc-f198-465f-8b76-aa7d0bdc4aa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:11.569434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-69547dbc-f198-465f-8b76-aa7d0bdc4aa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:11.583058 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69547dbc-f198-465f-8b76-aa7d0bdc4aa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:11.584226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69547dbc-f198-465f-8b76-aa7d0bdc4aa9 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:11.596898 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1380/2756] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:14:12.886130 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992ff44-9392-460c-ba8a-ae971c1eb6da service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:12.895579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992ff44-9392-460c-ba8a-ae971c1eb6da service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:12.900757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0aade-6921-47a3-bcae-99c389f07f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:12.901412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0aade-6921-47a3-bcae-99c389f07f72 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:12.901695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0aade-6921-47a3-bcae-99c389f07f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:12.901970 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0aade-6921-47a3-bcae-99c389f07f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:12.904090 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-41c0aade-6921-47a3-bcae-99c389f07f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:12.972413 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0aade-6921-47a3-bcae-99c389f07f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:12.973938 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0aade-6921-47a3-bcae-99c389f07f72 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:12.984299 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1377/2757] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:14:16.432985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69547dbc-f198-465f-8b76-aa7d0bdc4aa9 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:16.435038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-69547dbc-f198-465f-8b76-aa7d0bdc4aa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:16.518495 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69547dbc-f198-465f-8b76-aa7d0bdc4aa9 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:16.523515 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1f851-5690-4c1c-8060-9ce9da0b9e96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:16.524286 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1f851-5690-4c1c-8060-9ce9da0b9e96 service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:16.524536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1f851-5690-4c1c-8060-9ce9da0b9e96 service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:16.525027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1f851-5690-4c1c-8060-9ce9da0b9e96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:16.526823 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-87e1f851-5690-4c1c-8060-9ce9da0b9e96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:16.538874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1f851-5690-4c1c-8060-9ce9da0b9e96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:16.540709 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1f851-5690-4c1c-8060-9ce9da0b9e96 service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:16.552995 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1381/2758] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:14:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:14:19.538083 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0aade-6921-47a3-bcae-99c389f07f72 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:19.539775 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-41c0aade-6921-47a3-bcae-99c389f07f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:19.552133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0aade-6921-47a3-bcae-99c389f07f72 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:19.557973 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:19.558884 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:19.559186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:19.559441 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] PATH_INFO: `/v3/users/e3223a1d1dc1445c8b4aa118eb9747fd` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:19.560784 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3223a1d1dc1445c8b4aa118eb9747fd)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:19.562723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:19.611725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3223a1d1dc1445c8b4aa118eb9747fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:19.612462 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3223a1d1dc1445c8b4aa118eb9747fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:19.619524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3223a1d1dc1445c8b4aa118eb9747fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:19.620183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3223a1d1dc1445c8b4aa118eb9747fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:19.631933 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3223a1d1dc1445c8b4aa118eb9747fd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:19.645096 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1378/2759] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:14:19 2022] DELETE /identity/v3/users/e3223a1d1dc1445c8b4aa118eb9747fd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:14:20.096129 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1f851-5690-4c1c-8060-9ce9da0b9e96 service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:20.107484 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1f851-5690-4c1c-8060-9ce9da0b9e96 service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:20.164576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:20.165458 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:20.165738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:20.166008 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] PATH_INFO: `/v3/projects/3a26558aef284a9cbb955c7cdaa58e4e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:20.167315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a26558aef284a9cbb955c7cdaa58e4e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:20.169289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:20.186826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:14:20.216246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a26558aef284a9cbb955c7cdaa58e4e', 'request_id': 'req-4b31935c-8d77-4f11-81fd-170e354bb809'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.222170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a26558aef284a9cbb955c7cdaa58e4e', 'request_id': 'req-4b31935c-8d77-4f11-81fd-170e354bb809'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.230826 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a26558aef284a9cbb955c7cdaa58e4e', 'request_id': 'req-4b31935c-8d77-4f11-81fd-170e354bb809'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.242149 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] The token cache is being invalidate because project 3a26558aef284a9cbb955c7cdaa58e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:14:20.242516 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.243207 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.244956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1382/2760] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:14:20 2022] DELETE /identity/v3/projects/3a26558aef284a9cbb955c7cdaa58e4e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:14:20.249187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:20.250634 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:20.263702 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18f60c-e2a3-484a-af93-a71d29bd6abf admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:20.310912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:14:20.320818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:20.322747 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:20.323066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:20.323471 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] PATH_INFO: `/v3/users/0862fcd84e21434389874fab02cd92e3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:20.324911 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0862fcd84e21434389874fab02cd92e3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:20.326827 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:20.379698 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0862fcd84e21434389874fab02cd92e3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.380453 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0862fcd84e21434389874fab02cd92e3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.388131 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0862fcd84e21434389874fab02cd92e3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.388782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0862fcd84e21434389874fab02cd92e3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.400757 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0862fcd84e21434389874fab02cd92e3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.415231 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1379/2761] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:14:20 2022] DELETE /identity/v3/users/0862fcd84e21434389874fab02cd92e3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:14:20.606707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:20.608624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:20.623941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31935c-8d77-4f11-81fd-170e354bb809 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:20.683075 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:20.683965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:20.684270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:20.684544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] PATH_INFO: `/v3/projects/dfabd2f957cf47598ef20e4293b87e14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:20.685779 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfabd2f957cf47598ef20e4293b87e14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:20.687755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:20.736309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfabd2f957cf47598ef20e4293b87e14', 'request_id': 'req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.742498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfabd2f957cf47598ef20e4293b87e14', 'request_id': 'req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.750951 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfabd2f957cf47598ef20e4293b87e14', 'request_id': 'req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.761474 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] The token cache is being invalidate because project dfabd2f957cf47598ef20e4293b87e14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:14:20.761766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.762438 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:20.764056 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1383/2762] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:14:20 2022] DELETE /identity/v3/projects/dfabd2f957cf47598ef20e4293b87e14 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:14:30.411876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:14:30.758454 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=68,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:14:46.876976 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:46.878667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:46.941263 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf1970-176e-4c7d-99ac-14a22a8a3265 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:46.990330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92622276-10c8-4d70-a9b2-733467f7b566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:46.990977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92622276-10c8-4d70-a9b2-733467f7b566 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:46.991254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92622276-10c8-4d70-a9b2-733467f7b566 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:46.991527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92622276-10c8-4d70-a9b2-733467f7b566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:46.993542 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-92622276-10c8-4d70-a9b2-733467f7b566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:47.059864 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92622276-10c8-4d70-a9b2-733467f7b566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:47.061275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92622276-10c8-4d70-a9b2-733467f7b566 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:47.071328 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1380/2763] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:14:50.339536 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:50.341136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:50.355203 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc2354-60e4-46ac-9bac-37b1f3d0a67d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:50.362363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:50.363325 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:50.363605 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:50.363879 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] PATH_INFO: `/v3/users/68f6997db7c141529c37cc64955df779` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:50.365294 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68f6997db7c141529c37cc64955df779)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:50.367715 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:50.423440 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68f6997db7c141529c37cc64955df779'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:50.431479 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68f6997db7c141529c37cc64955df779'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:50.432285 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68f6997db7c141529c37cc64955df779'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:50.443424 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68f6997db7c141529c37cc64955df779'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:50.456236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e6f990c-6d1f-4f69-b92e-32db8168de64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68f6997db7c141529c37cc64955df779'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:50.462649 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1384/2764] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:14:50 2022] DELETE /identity/v3/users/68f6997db7c141529c37cc64955df779 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:14:50.647764 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92622276-10c8-4d70-a9b2-733467f7b566 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:50.657869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92622276-10c8-4d70-a9b2-733467f7b566 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:50.734946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:50.736218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:50.736557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:50.737079 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] PATH_INFO: `/v3/projects/b85a8791815f40f98ed1e81e58b01fd6` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:50.738467 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b85a8791815f40f98ed1e81e58b01fd6)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:50.743010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:50.750223 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:14:50.796125 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b85a8791815f40f98ed1e81e58b01fd6', 'request_id': 'req-0d66c097-c428-46d4-ac7e-2b8c189be7b3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:50.802293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b85a8791815f40f98ed1e81e58b01fd6', 'request_id': 'req-0d66c097-c428-46d4-ac7e-2b8c189be7b3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:50.810720 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b85a8791815f40f98ed1e81e58b01fd6', 'request_id': 'req-0d66c097-c428-46d4-ac7e-2b8c189be7b3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:50.821939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] The token cache is being invalidate because project b85a8791815f40f98ed1e81e58b01fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:14:50.822235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:50.822939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0d66c097-c428-46d4-ac7e-2b8c189be7b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:14:50.824709 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1381/2765] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:14:50 2022] DELETE /identity/v3/projects/b85a8791815f40f98ed1e81e58b01fd6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:14:50.830255 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079848de-edbc-4d67-8511-da524bfb1b0f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:50.830564 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079848de-edbc-4d67-8511-da524bfb1b0f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:50.830982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079848de-edbc-4d67-8511-da524bfb1b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:50.854112 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-079848de-edbc-4d67-8511-da524bfb1b0f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:14:50.855316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-079848de-edbc-4d67-8511-da524bfb1b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:50.918487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1385/2766] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:14:50.927078 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045edc3-0070-46c0-96a4-e0283ea6aa9f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:50.927474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045edc3-0070-46c0-96a4-e0283ea6aa9f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:50.927810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045edc3-0070-46c0-96a4-e0283ea6aa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:50.948753 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2045edc3-0070-46c0-96a4-e0283ea6aa9f None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:14:50.950012 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2045edc3-0070-46c0-96a4-e0283ea6aa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:50.959191 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1382/2767] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:14:50.963690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079848de-edbc-4d67-8511-da524bfb1b0f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:50.974877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079848de-edbc-4d67-8511-da524bfb1b0f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:50.979876 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b478e-4ae9-4006-8256-2a1141e2f5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:50.980593 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b478e-4ae9-4006-8256-2a1141e2f5ab admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:50.980866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b478e-4ae9-4006-8256-2a1141e2f5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:50.981150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b478e-4ae9-4006-8256-2a1141e2f5ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:50.981899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b478e-4ae9-4006-8256-2a1141e2f5ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:14:50.982210 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b478e-4ae9-4006-8256-2a1141e2f5ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:50.983849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b478e-4ae9-4006-8256-2a1141e2f5ab admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:50.997320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1386/2768] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:14:51.001816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045edc3-0070-46c0-96a4-e0283ea6aa9f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:51.012549 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045edc3-0070-46c0-96a4-e0283ea6aa9f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:51.018778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c362cd8-cf92-4a15-bd38-23f3d41d9e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:51.019448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c362cd8-cf92-4a15-bd38-23f3d41d9e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:51.019727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c362cd8-cf92-4a15-bd38-23f3d41d9e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:51.019995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c362cd8-cf92-4a15-bd38-23f3d41d9e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:51.020735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c362cd8-cf92-4a15-bd38-23f3d41d9e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:51.022664 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c362cd8-cf92-4a15-bd38-23f3d41d9e36 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:51.057579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1383/2769] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:14:51 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:14:51.061694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b478e-4ae9-4006-8256-2a1141e2f5ab admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:51.076001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b478e-4ae9-4006-8256-2a1141e2f5ab admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:51.142861 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca6fcd-c338-4176-814e-2ebfa6bc7bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:51.144067 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca6fcd-c338-4176-814e-2ebfa6bc7bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:51.144387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca6fcd-c338-4176-814e-2ebfa6bc7bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:51.144668 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca6fcd-c338-4176-814e-2ebfa6bc7bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:51.145371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca6fcd-c338-4176-814e-2ebfa6bc7bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:51.147327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca6fcd-c338-4176-814e-2ebfa6bc7bc1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:51.154069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:14:51.172244 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1387/2770] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:14:51 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:14:51.176766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c362cd8-cf92-4a15-bd38-23f3d41d9e36 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:51.187403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c362cd8-cf92-4a15-bd38-23f3d41d9e36 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:51.193544 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c5462-8dc5-4442-8d5b-bbb28e2b6e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:51.194214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c5462-8dc5-4442-8d5b-bbb28e2b6e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:51.194498 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c5462-8dc5-4442-8d5b-bbb28e2b6e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:51.194769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c5462-8dc5-4442-8d5b-bbb28e2b6e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:51.195508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c5462-8dc5-4442-8d5b-bbb28e2b6e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:51.197145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c5462-8dc5-4442-8d5b-bbb28e2b6e85 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:51.208166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1384/2771] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:14:51.212297 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca6fcd-c338-4176-814e-2ebfa6bc7bc1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:51.222782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca6fcd-c338-4176-814e-2ebfa6bc7bc1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:51.228074 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02695f2d-0a08-48b9-99ea-e0fa33915b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:51.228762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02695f2d-0a08-48b9-99ea-e0fa33915b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:51.229043 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02695f2d-0a08-48b9-99ea-e0fa33915b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:51.229316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02695f2d-0a08-48b9-99ea-e0fa33915b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:51.230073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02695f2d-0a08-48b9-99ea-e0fa33915b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:51.231684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02695f2d-0a08-48b9-99ea-e0fa33915b4b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:51.242109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1388/2772] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:14:51.246221 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c5462-8dc5-4442-8d5b-bbb28e2b6e85 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:51.256474 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c5462-8dc5-4442-8d5b-bbb28e2b6e85 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:51.261279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61e695-67f6-4297-bfa8-9defa4adebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:51.262158 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61e695-67f6-4297-bfa8-9defa4adebda admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:51.262436 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61e695-67f6-4297-bfa8-9defa4adebda admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:51.262713 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61e695-67f6-4297-bfa8-9defa4adebda admin admin] PATH_INFO: `/v3/projects/6330a6c7b6a74599ad9c4ca8ebf8fa75/users/c02cb425c9904c238b9ae72049722c55/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:51.277633 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61e695-67f6-4297-bfa8-9defa4adebda admin admin] RBAC: Authorizing `identity:create_grant(project_id=6330a6c7b6a74599ad9c4ca8ebf8fa75, user_id=c02cb425c9904c238b9ae72049722c55, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:51.280311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61e695-67f6-4297-bfa8-9defa4adebda admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:51.296872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1385/2773] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:14:51 2022] PUT /identity/v3/projects/6330a6c7b6a74599ad9c4ca8ebf8fa75/users/c02cb425c9904c238b9ae72049722c55/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:14:51.307671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2618a65-0059-44dd-a6b9-19aeebfb869f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:51.307960 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2618a65-0059-44dd-a6b9-19aeebfb869f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:51.308268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2618a65-0059-44dd-a6b9-19aeebfb869f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:51.337203 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e2618a65-0059-44dd-a6b9-19aeebfb869f None None] MFA Rules not processed for user `c02cb425c9904c238b9ae72049722c55`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:14:51.338382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e2618a65-0059-44dd-a6b9-19aeebfb869f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:51.417888 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1389/2774] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:14:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:14:51.430810 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61e695-67f6-4297-bfa8-9defa4adebda admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:14:51.433941 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4f61e695-67f6-4297-bfa8-9defa4adebda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:14:51.515385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61e695-67f6-4297-bfa8-9defa4adebda admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:14:51.525614 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acda0af-a91e-4f6e-8b6a-b9ebaf34f264 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acb3ad858e4848b9a325c72c7328c863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:14:51.526533 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acda0af-a91e-4f6e-8b6a-b9ebaf34f264 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:14:51.526935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acda0af-a91e-4f6e-8b6a-b9ebaf34f264 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:14:51.527332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acda0af-a91e-4f6e-8b6a-b9ebaf34f264 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:14:51.539060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acda0af-a91e-4f6e-8b6a-b9ebaf34f264 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:14:51.540682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acda0af-a91e-4f6e-8b6a-b9ebaf34f264 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:14:51.550832 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1386/2775] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:14:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:14:59.986236 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2618a65-0059-44dd-a6b9-19aeebfb869f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:00.053320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2618a65-0059-44dd-a6b9-19aeebfb869f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:00.061986 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:00.063282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:00.063648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:00.063982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] PATH_INFO: `/v3/users/c02cb425c9904c238b9ae72049722c55` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:00.065965 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c02cb425c9904c238b9ae72049722c55)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:00.069051 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:00.120410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c02cb425c9904c238b9ae72049722c55'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:00.127356 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c02cb425c9904c238b9ae72049722c55'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:00.128044 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c02cb425c9904c238b9ae72049722c55'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:00.138082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c02cb425c9904c238b9ae72049722c55'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:00.143676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:00.152327 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c02cb425c9904c238b9ae72049722c55'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:00.158695 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1390/2776] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:14:59 2022] DELETE /identity/v3/users/c02cb425c9904c238b9ae72049722c55 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:00.163382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425418b4-8806-455e-a799-ac4212c7eddb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:00.163877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425418b4-8806-455e-a799-ac4212c7eddb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:00.163982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425418b4-8806-455e-a799-ac4212c7eddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:00.182302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-425418b4-8806-455e-a799-ac4212c7eddb None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:00.183527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-425418b4-8806-455e-a799-ac4212c7eddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:00.197322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:00.247220 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1387/2777] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:00.257371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:00.258827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:00.318544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94b5e0-4ea5-4359-94e6-3f54ceb5904f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:00.322579 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4899aa-dbd1-4ef2-be8f-d19cab231291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:00.323224 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4899aa-dbd1-4ef2-be8f-d19cab231291 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:00.323523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4899aa-dbd1-4ef2-be8f-d19cab231291 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:00.323775 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4899aa-dbd1-4ef2-be8f-d19cab231291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:00.333178 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4899aa-dbd1-4ef2-be8f-d19cab231291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:00.334797 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4899aa-dbd1-4ef2-be8f-d19cab231291 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:00.344870 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1391/2778] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:00.617175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425418b4-8806-455e-a799-ac4212c7eddb None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:00.618629 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-425418b4-8806-455e-a799-ac4212c7eddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:00.630778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425418b4-8806-455e-a799-ac4212c7eddb None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:00.636756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:00.637761 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:00.638082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:00.638538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] PATH_INFO: `/v3/projects/6330a6c7b6a74599ad9c4ca8ebf8fa75` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:00.639951 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6330a6c7b6a74599ad9c4ca8ebf8fa75)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:00.642387 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:00.693876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6330a6c7b6a74599ad9c4ca8ebf8fa75', 'request_id': 'req-359871d4-4524-4459-9c72-e5a1c24edcb7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:00.699975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6330a6c7b6a74599ad9c4ca8ebf8fa75', 'request_id': 'req-359871d4-4524-4459-9c72-e5a1c24edcb7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:00.709370 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6330a6c7b6a74599ad9c4ca8ebf8fa75', 'request_id': 'req-359871d4-4524-4459-9c72-e5a1c24edcb7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:00.721081 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] The token cache is being invalidate because project 6330a6c7b6a74599ad9c4ca8ebf8fa75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:15:00.721368 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:00.722017 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-359871d4-4524-4459-9c72-e5a1c24edcb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:00.723593 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1388/2779] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:15:00 2022] DELETE /identity/v3/projects/6330a6c7b6a74599ad9c4ca8ebf8fa75 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:00.728663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fc5b3-9a1d-47a7-9a46-a82c0cfb40e7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:00.728952 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fc5b3-9a1d-47a7-9a46-a82c0cfb40e7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:00.729226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fc5b3-9a1d-47a7-9a46-a82c0cfb40e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:00.746732 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-da1fc5b3-9a1d-47a7-9a46-a82c0cfb40e7 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:00.747928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-da1fc5b3-9a1d-47a7-9a46-a82c0cfb40e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:00.810123 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1392/2780] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:00.818194 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa89d0-df20-4fbc-b570-715c7a8adf64 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:00.818514 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa89d0-df20-4fbc-b570-715c7a8adf64 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:00.818780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa89d0-df20-4fbc-b570-715c7a8adf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:00.839843 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3cfa89d0-df20-4fbc-b570-715c7a8adf64 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:00.841103 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3cfa89d0-df20-4fbc-b570-715c7a8adf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:00.851876 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1389/2781] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:00.856422 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fc5b3-9a1d-47a7-9a46-a82c0cfb40e7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:00.866907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fc5b3-9a1d-47a7-9a46-a82c0cfb40e7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:00.873455 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34df968-be5c-4410-b908-4b44dea67eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:00.874115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34df968-be5c-4410-b908-4b44dea67eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:00.874391 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34df968-be5c-4410-b908-4b44dea67eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:00.874662 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34df968-be5c-4410-b908-4b44dea67eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:00.875381 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34df968-be5c-4410-b908-4b44dea67eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:15:00.875685 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34df968-be5c-4410-b908-4b44dea67eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:00.877365 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34df968-be5c-4410-b908-4b44dea67eb2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:00.889939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1393/2782] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:15:00.894235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa89d0-df20-4fbc-b570-715c7a8adf64 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:00.903766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa89d0-df20-4fbc-b570-715c7a8adf64 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:00.909975 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62726ac-9fb0-4fb9-abea-2bf1cd25805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:00.910628 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62726ac-9fb0-4fb9-abea-2bf1cd25805d admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:00.910907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62726ac-9fb0-4fb9-abea-2bf1cd25805d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:00.911172 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62726ac-9fb0-4fb9-abea-2bf1cd25805d admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:00.911859 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62726ac-9fb0-4fb9-abea-2bf1cd25805d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:00.913871 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62726ac-9fb0-4fb9-abea-2bf1cd25805d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:00.945218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1390/2783] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:15:00 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:00.951344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34df968-be5c-4410-b908-4b44dea67eb2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:00.966961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34df968-be5c-4410-b908-4b44dea67eb2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:01.026914 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f95ec-5413-4702-b2ea-3b2202aaaca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:01.027580 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f95ec-5413-4702-b2ea-3b2202aaaca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.027845 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f95ec-5413-4702-b2ea-3b2202aaaca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.028145 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f95ec-5413-4702-b2ea-3b2202aaaca5 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.028827 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f95ec-5413-4702-b2ea-3b2202aaaca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:01.030760 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f95ec-5413-4702-b2ea-3b2202aaaca5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:01.053329 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1394/2784] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:15:00 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:01.058260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62726ac-9fb0-4fb9-abea-2bf1cd25805d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:01.075415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62726ac-9fb0-4fb9-abea-2bf1cd25805d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:01.084807 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687a22e-f2e9-4fd9-8861-7172ef3b81a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:01.085769 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687a22e-f2e9-4fd9-8861-7172ef3b81a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.086057 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687a22e-f2e9-4fd9-8861-7172ef3b81a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.086332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687a22e-f2e9-4fd9-8861-7172ef3b81a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.087059 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687a22e-f2e9-4fd9-8861-7172ef3b81a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:01.088723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687a22e-f2e9-4fd9-8861-7172ef3b81a7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:01.099582 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1391/2785] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:01.104098 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f95ec-5413-4702-b2ea-3b2202aaaca5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:01.114848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f95ec-5413-4702-b2ea-3b2202aaaca5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:01.121588 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f004828-3b6e-4c05-a5e7-3834ab9d44e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:01.122250 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f004828-3b6e-4c05-a5e7-3834ab9d44e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.122537 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f004828-3b6e-4c05-a5e7-3834ab9d44e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.122813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f004828-3b6e-4c05-a5e7-3834ab9d44e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.123538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f004828-3b6e-4c05-a5e7-3834ab9d44e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:01.125226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f004828-3b6e-4c05-a5e7-3834ab9d44e0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:01.126456 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:01.137419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1395/2786] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:01.142171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687a22e-f2e9-4fd9-8861-7172ef3b81a7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:01.153077 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687a22e-f2e9-4fd9-8861-7172ef3b81a7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:01.158206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e10c2-2bc7-460f-ad82-0f8a3c4c9050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:01.159114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e10c2-2bc7-460f-ad82-0f8a3c4c9050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.159397 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e10c2-2bc7-460f-ad82-0f8a3c4c9050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.159686 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e10c2-2bc7-460f-ad82-0f8a3c4c9050 admin admin] PATH_INFO: `/v3/projects/99300b33ccb44adc8177f2e3804038ac/users/b34516e7b21b4a108ef78e33b0c64bac/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.176303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e10c2-2bc7-460f-ad82-0f8a3c4c9050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99300b33ccb44adc8177f2e3804038ac, user_id=b34516e7b21b4a108ef78e33b0c64bac, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:01.179758 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e10c2-2bc7-460f-ad82-0f8a3c4c9050 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:01.195144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1392/2787] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:01 2022] PUT /identity/v3/projects/99300b33ccb44adc8177f2e3804038ac/users/b34516e7b21b4a108ef78e33b0c64bac/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:01.206803 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56683c8c-2ff5-42ec-a451-5f83785cd20e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.207063 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56683c8c-2ff5-42ec-a451-5f83785cd20e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.207340 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56683c8c-2ff5-42ec-a451-5f83785cd20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.236912 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-56683c8c-2ff5-42ec-a451-5f83785cd20e None None] MFA Rules not processed for user `b34516e7b21b4a108ef78e33b0c64bac`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:01.238135 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-56683c8c-2ff5-42ec-a451-5f83785cd20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:01.303410 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1396/2788] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:01.308686 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e10c2-2bc7-460f-ad82-0f8a3c4c9050 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:01.353966 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:01.377239 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e10c2-2bc7-460f-ad82-0f8a3c4c9050 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:01.382718 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aaaaa8-0d4c-4ce4-8f3d-b02825bb8a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:01.383378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aaaaa8-0d4c-4ce4-8f3d-b02825bb8a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.383662 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aaaaa8-0d4c-4ce4-8f3d-b02825bb8a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.383939 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aaaaa8-0d4c-4ce4-8f3d-b02825bb8a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.384667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aaaaa8-0d4c-4ce4-8f3d-b02825bb8a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:01.386694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aaaaa8-0d4c-4ce4-8f3d-b02825bb8a8a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:01.416689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1393/2789] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:15:01 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:01.421304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56683c8c-2ff5-42ec-a451-5f83785cd20e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:01.433010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56683c8c-2ff5-42ec-a451-5f83785cd20e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:01.496009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f2dd1-243c-4719-b7e4-2061610ec7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:01.496634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f2dd1-243c-4719-b7e4-2061610ec7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.496910 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f2dd1-243c-4719-b7e4-2061610ec7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.497184 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f2dd1-243c-4719-b7e4-2061610ec7d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.497867 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f2dd1-243c-4719-b7e4-2061610ec7d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:01.499794 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f2dd1-243c-4719-b7e4-2061610ec7d2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:01.526782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1397/2790] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:15:01 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:01.532332 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aaaaa8-0d4c-4ce4-8f3d-b02825bb8a8a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:01.543537 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aaaaa8-0d4c-4ce4-8f3d-b02825bb8a8a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:01.549113 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15111e4c-706a-47eb-b627-0590211e2cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:01.549780 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15111e4c-706a-47eb-b627-0590211e2cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.550064 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15111e4c-706a-47eb-b627-0590211e2cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.550323 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15111e4c-706a-47eb-b627-0590211e2cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.551056 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15111e4c-706a-47eb-b627-0590211e2cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:01.552702 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15111e4c-706a-47eb-b627-0590211e2cf2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:01.692133 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1394/2791] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:01.696814 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f2dd1-243c-4719-b7e4-2061610ec7d2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:01.707743 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f2dd1-243c-4719-b7e4-2061610ec7d2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:01.714150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57372750-68f4-43c0-a8f6-a45274ca8fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:01.715344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57372750-68f4-43c0-a8f6-a45274ca8fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.715622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57372750-68f4-43c0-a8f6-a45274ca8fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.715895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57372750-68f4-43c0-a8f6-a45274ca8fb8 admin admin] PATH_INFO: `/v3/projects/dec83738517541129fcbccb56e46262d/users/c8bd12f843bb47e5b4fad965dc7e4b1c/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.733024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57372750-68f4-43c0-a8f6-a45274ca8fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec83738517541129fcbccb56e46262d, user_id=c8bd12f843bb47e5b4fad965dc7e4b1c, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:01.735684 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57372750-68f4-43c0-a8f6-a45274ca8fb8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:01.750925 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1398/2792] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:01 2022] PUT /identity/v3/projects/dec83738517541129fcbccb56e46262d/users/c8bd12f843bb47e5b4fad965dc7e4b1c/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:01.754986 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15111e4c-706a-47eb-b627-0590211e2cf2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:01.765576 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15111e4c-706a-47eb-b627-0590211e2cf2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:01.825594 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285fadc-6b82-490b-bcf4-25871d793575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:01.826268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285fadc-6b82-490b-bcf4-25871d793575 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.826545 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285fadc-6b82-490b-bcf4-25871d793575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.826811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285fadc-6b82-490b-bcf4-25871d793575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.827546 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285fadc-6b82-490b-bcf4-25871d793575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:01.829183 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285fadc-6b82-490b-bcf4-25871d793575 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:01.839461 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1395/2793] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:01.843977 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57372750-68f4-43c0-a8f6-a45274ca8fb8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:01.854725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57372750-68f4-43c0-a8f6-a45274ca8fb8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:01.862996 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9cda4-53b4-4aee-bfaa-d2da0f0d1ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:01.864444 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9cda4-53b4-4aee-bfaa-d2da0f0d1ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.864832 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9cda4-53b4-4aee-bfaa-d2da0f0d1ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.865185 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9cda4-53b4-4aee-bfaa-d2da0f0d1ac6 admin admin] PATH_INFO: `/v3/projects/dec83738517541129fcbccb56e46262d/users/c8bd12f843bb47e5b4fad965dc7e4b1c/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.868128 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9cda4-53b4-4aee-bfaa-d2da0f0d1ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec83738517541129fcbccb56e46262d, user_id=c8bd12f843bb47e5b4fad965dc7e4b1c, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:01.870819 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9cda4-53b4-4aee-bfaa-d2da0f0d1ac6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:01.882538 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1399/2794] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:01 2022] PUT /identity/v3/projects/dec83738517541129fcbccb56e46262d/users/c8bd12f843bb47e5b4fad965dc7e4b1c/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:01.893719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567fbd6-c782-4a9b-8f39-ec3b8b631295 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:01.894034 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567fbd6-c782-4a9b-8f39-ec3b8b631295 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:01.894388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567fbd6-c782-4a9b-8f39-ec3b8b631295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:01.922076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1567fbd6-c782-4a9b-8f39-ec3b8b631295 None None] MFA Rules not processed for user `c8bd12f843bb47e5b4fad965dc7e4b1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:01.923277 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1567fbd6-c782-4a9b-8f39-ec3b8b631295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:01.993254 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1396/2795] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:02.009069 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9cda4-53b4-4aee-bfaa-d2da0f0d1ac6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:02.011899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-51e9cda4-53b4-4aee-bfaa-d2da0f0d1ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:02.060423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:02.082494 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9cda4-53b4-4aee-bfaa-d2da0f0d1ac6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:02.088450 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd4b7b-6938-46b7-810e-4517e39b391b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:02.089109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd4b7b-6938-46b7-810e-4517e39b391b service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:02.089389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd4b7b-6938-46b7-810e-4517e39b391b service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:02.089658 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd4b7b-6938-46b7-810e-4517e39b391b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:02.099027 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd4b7b-6938-46b7-810e-4517e39b391b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:02.100542 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd4b7b-6938-46b7-810e-4517e39b391b service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:02.110485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1400/2796] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:02.192804 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567fbd6-c782-4a9b-8f39-ec3b8b631295 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:02.203600 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567fbd6-c782-4a9b-8f39-ec3b8b631295 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:02.207638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daace3dd-195b-4393-9473-f08289e40ecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:02.208320 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daace3dd-195b-4393-9473-f08289e40ecb service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:02.208607 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daace3dd-195b-4393-9473-f08289e40ecb service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:02.208880 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daace3dd-195b-4393-9473-f08289e40ecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:02.244316 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:02.276927 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daace3dd-195b-4393-9473-f08289e40ecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:02.278303 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daace3dd-195b-4393-9473-f08289e40ecb service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:02.287797 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1397/2797] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:02 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:12.105993 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=9 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:12.283465 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=13 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:12.617681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd4b7b-6938-46b7-810e-4517e39b391b service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:12.683816 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd4b7b-6938-46b7-810e-4517e39b391b service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:12.688987 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:12.689875 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:12.690154 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:12.690426 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] PATH_INFO: `/v3/users/b34516e7b21b4a108ef78e33b0c64bac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:12.691637 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b34516e7b21b4a108ef78e33b0c64bac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:12.693608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:12.740470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b34516e7b21b4a108ef78e33b0c64bac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:12.747174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b34516e7b21b4a108ef78e33b0c64bac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:12.747867 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b34516e7b21b4a108ef78e33b0c64bac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:12.756569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b34516e7b21b4a108ef78e33b0c64bac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:12.768675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b34516e7b21b4a108ef78e33b0c64bac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:12.774527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1401/2798] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:15:12 2022] DELETE /identity/v3/users/b34516e7b21b4a108ef78e33b0c64bac => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:12.779107 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae93de0-52d2-4495-b400-7c38fa256db6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:12.779491 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae93de0-52d2-4495-b400-7c38fa256db6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:12.779783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae93de0-52d2-4495-b400-7c38fa256db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:12.797863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-dae93de0-52d2-4495-b400-7c38fa256db6 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:12.799076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dae93de0-52d2-4495-b400-7c38fa256db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:12.863995 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1398/2799] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:12.874230 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:12.875791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:12.934346 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ef84b-fa0d-4db1-9a1b-6e5f8ef1774e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:12.938309 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acaae9-1cf2-48b3-87bc-d0a86e674a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:12.938957 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acaae9-1cf2-48b3-87bc-d0a86e674a7e service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:12.939237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acaae9-1cf2-48b3-87bc-d0a86e674a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:12.939513 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acaae9-1cf2-48b3-87bc-d0a86e674a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:12.949924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acaae9-1cf2-48b3-87bc-d0a86e674a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:12.952453 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acaae9-1cf2-48b3-87bc-d0a86e674a7e service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:12.967272 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1402/2800] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:13.266346 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae93de0-52d2-4495-b400-7c38fa256db6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:13.267900 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dae93de0-52d2-4495-b400-7c38fa256db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:13.280412 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae93de0-52d2-4495-b400-7c38fa256db6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:13.285215 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:13.286082 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:13.286356 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:13.286610 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] PATH_INFO: `/v3/projects/99300b33ccb44adc8177f2e3804038ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:13.287823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99300b33ccb44adc8177f2e3804038ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:13.289796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:13.339212 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99300b33ccb44adc8177f2e3804038ac', 'request_id': 'req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.345286 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99300b33ccb44adc8177f2e3804038ac', 'request_id': 'req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.353886 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99300b33ccb44adc8177f2e3804038ac', 'request_id': 'req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.364528 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] The token cache is being invalidate because project 99300b33ccb44adc8177f2e3804038ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:15:13.364809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.365463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.367046 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1399/2801] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:15:13 2022] DELETE /identity/v3/projects/99300b33ccb44adc8177f2e3804038ac => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:13.371205 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acaae9-1cf2-48b3-87bc-d0a86e674a7e service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:13.381031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acaae9-1cf2-48b3-87bc-d0a86e674a7e service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:13.401089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:13.448251 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:13.449153 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:13.449514 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:13.449860 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] PATH_INFO: `/v3/users/c8bd12f843bb47e5b4fad965dc7e4b1c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:13.451625 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8bd12f843bb47e5b4fad965dc7e4b1c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:13.453776 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:13.513089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8bd12f843bb47e5b4fad965dc7e4b1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.520558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8bd12f843bb47e5b4fad965dc7e4b1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.521254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8bd12f843bb47e5b4fad965dc7e4b1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.531969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8bd12f843bb47e5b4fad965dc7e4b1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.543670 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2100ee31-1a69-45e8-96ca-6f9996281b40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8bd12f843bb47e5b4fad965dc7e4b1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.549883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1403/2802] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:15:13 2022] DELETE /identity/v3/users/c8bd12f843bb47e5b4fad965dc7e4b1c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:13.841623 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:13.845858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:13.860872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ff01e-a07c-4cee-80c2-c0f5a7d48b61 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:13.921608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:13.922418 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:13.922704 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:13.922985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] PATH_INFO: `/v3/projects/dec83738517541129fcbccb56e46262d` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:13.924330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dec83738517541129fcbccb56e46262d)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:13.926298 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:13.961946 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:13.985042 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dec83738517541129fcbccb56e46262d', 'request_id': 'req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:13.991414 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dec83738517541129fcbccb56e46262d', 'request_id': 'req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:14.000312 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dec83738517541129fcbccb56e46262d', 'request_id': 'req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:14.011138 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] The token cache is being invalidate because project dec83738517541129fcbccb56e46262d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:15:14.011421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:14.012055 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-bf4225e3-55c6-4b45-8c0b-44996b8cecf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:14.013751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1400/2803] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:15:13 2022] DELETE /identity/v3/projects/dec83738517541129fcbccb56e46262d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:14.018824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74fd78-99c4-4d21-8c7e-ff5d4e014afd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.019105 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74fd78-99c4-4d21-8c7e-ff5d4e014afd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.019382 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74fd78-99c4-4d21-8c7e-ff5d4e014afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.037470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1f74fd78-99c4-4d21-8c7e-ff5d4e014afd None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:14.038706 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1f74fd78-99c4-4d21-8c7e-ff5d4e014afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:14.108490 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1404/2804] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:14.117943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012f1d9-ad6b-4064-8b40-c2ea0002b123 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.118403 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012f1d9-ad6b-4064-8b40-c2ea0002b123 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.118865 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012f1d9-ad6b-4064-8b40-c2ea0002b123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.146703 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3012f1d9-ad6b-4064-8b40-c2ea0002b123 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:14.147914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3012f1d9-ad6b-4064-8b40-c2ea0002b123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:14.157031 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1401/2805] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:14.161693 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74fd78-99c4-4d21-8c7e-ff5d4e014afd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:14.172746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74fd78-99c4-4d21-8c7e-ff5d4e014afd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:14.178559 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd995e-3836-4060-8936-f96ac11309ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:14.179237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd995e-3836-4060-8936-f96ac11309ad admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.179528 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd995e-3836-4060-8936-f96ac11309ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.179801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd995e-3836-4060-8936-f96ac11309ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.180572 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd995e-3836-4060-8936-f96ac11309ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:15:14.180885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd995e-3836-4060-8936-f96ac11309ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:14.182483 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd995e-3836-4060-8936-f96ac11309ad admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:14.195807 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1405/2806] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:15:14.200929 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012f1d9-ad6b-4064-8b40-c2ea0002b123 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:14.210788 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012f1d9-ad6b-4064-8b40-c2ea0002b123 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:14.216723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f4dea-bf31-484c-8266-4a3ab221400d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:14.217604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f4dea-bf31-484c-8266-4a3ab221400d admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.217907 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f4dea-bf31-484c-8266-4a3ab221400d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.218177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f4dea-bf31-484c-8266-4a3ab221400d admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.218872 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f4dea-bf31-484c-8266-4a3ab221400d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:14.220811 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f4dea-bf31-484c-8266-4a3ab221400d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:14.249369 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1402/2807] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:15:14 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:14.253885 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd995e-3836-4060-8936-f96ac11309ad admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:14.264498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd995e-3836-4060-8936-f96ac11309ad admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:14.290473 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:14.322707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b407733-7dab-4771-8a9f-4345bdd3b358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:14.323380 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b407733-7dab-4771-8a9f-4345bdd3b358 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.323660 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b407733-7dab-4771-8a9f-4345bdd3b358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.323930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b407733-7dab-4771-8a9f-4345bdd3b358 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.324663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b407733-7dab-4771-8a9f-4345bdd3b358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:14.326782 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b407733-7dab-4771-8a9f-4345bdd3b358 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:14.348163 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1406/2808] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:15:14 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 21:15:14.352485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f4dea-bf31-484c-8266-4a3ab221400d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:14.363276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f4dea-bf31-484c-8266-4a3ab221400d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:14.368091 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d4f1-7513-453c-9934-abc45d8b3e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:14.368778 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d4f1-7513-453c-9934-abc45d8b3e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.369060 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d4f1-7513-453c-9934-abc45d8b3e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.369347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d4f1-7513-453c-9934-abc45d8b3e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.370114 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0d4f1-7513-453c-9934-abc45d8b3e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:14.371715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0d4f1-7513-453c-9934-abc45d8b3e3a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:14.382171 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1403/2809] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:14.388387 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b407733-7dab-4771-8a9f-4345bdd3b358 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:14.403941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b407733-7dab-4771-8a9f-4345bdd3b358 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:14.410033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f94859-44e5-4aba-8dbc-aa08d3fb87c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:14.410710 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f94859-44e5-4aba-8dbc-aa08d3fb87c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.410988 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f94859-44e5-4aba-8dbc-aa08d3fb87c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.411241 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f94859-44e5-4aba-8dbc-aa08d3fb87c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.411971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f94859-44e5-4aba-8dbc-aa08d3fb87c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:14.413702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f94859-44e5-4aba-8dbc-aa08d3fb87c5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:14.425527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1407/2810] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:14.430468 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d4f1-7513-453c-9934-abc45d8b3e3a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:14.443092 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d4f1-7513-453c-9934-abc45d8b3e3a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:14.448960 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5749c-e30a-4c3a-9562-d34e4fc3e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:14.449860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5749c-e30a-4c3a-9562-d34e4fc3e59e admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.450139 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5749c-e30a-4c3a-9562-d34e4fc3e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.450415 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5749c-e30a-4c3a-9562-d34e4fc3e59e admin admin] PATH_INFO: `/v3/projects/84964bfccd4a4620babfe3e36066cc81/users/d3f0e3a15c3b47cebd56b083b296fef2/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.466391 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5749c-e30a-4c3a-9562-d34e4fc3e59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=84964bfccd4a4620babfe3e36066cc81, user_id=d3f0e3a15c3b47cebd56b083b296fef2, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:14.469179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5749c-e30a-4c3a-9562-d34e4fc3e59e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:14.484135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1404/2811] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:14 2022] PUT /identity/v3/projects/84964bfccd4a4620babfe3e36066cc81/users/d3f0e3a15c3b47cebd56b083b296fef2/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:14.548153 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198c048-0a64-4350-97eb-b301ebeb33fb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.548501 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198c048-0a64-4350-97eb-b301ebeb33fb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.549005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198c048-0a64-4350-97eb-b301ebeb33fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.582830 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3198c048-0a64-4350-97eb-b301ebeb33fb None None] MFA Rules not processed for user `d3f0e3a15c3b47cebd56b083b296fef2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:14.584042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3198c048-0a64-4350-97eb-b301ebeb33fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:14.665573 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1408/2812] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:14.672682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5749c-e30a-4c3a-9562-d34e4fc3e59e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:14.746719 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5749c-e30a-4c3a-9562-d34e4fc3e59e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:14.752421 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711543f-c289-4c3c-adb1-ebfc55a5518e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:14.753084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711543f-c289-4c3c-adb1-ebfc55a5518e admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.753366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711543f-c289-4c3c-adb1-ebfc55a5518e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.753619 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711543f-c289-4c3c-adb1-ebfc55a5518e admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.754318 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711543f-c289-4c3c-adb1-ebfc55a5518e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:14.756269 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711543f-c289-4c3c-adb1-ebfc55a5518e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:14.785835 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1405/2813] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:15:14 2022] POST /identity/v3/projects => generated 391 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:14.791747 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198c048-0a64-4350-97eb-b301ebeb33fb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:14.804767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198c048-0a64-4350-97eb-b301ebeb33fb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:14.863971 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e6aec-e406-4e40-9412-0bc5ca95f73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:14.864681 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e6aec-e406-4e40-9412-0bc5ca95f73b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.864964 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e6aec-e406-4e40-9412-0bc5ca95f73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.865235 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e6aec-e406-4e40-9412-0bc5ca95f73b admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.865931 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e6aec-e406-4e40-9412-0bc5ca95f73b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:14.867877 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e6aec-e406-4e40-9412-0bc5ca95f73b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:14.890203 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1409/2814] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:15:14 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:14.894366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711543f-c289-4c3c-adb1-ebfc55a5518e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:14.905401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711543f-c289-4c3c-adb1-ebfc55a5518e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:14.910532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11d123-abc7-4c13-97fa-e524312c44b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:14.911195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d123-abc7-4c13-97fa-e524312c44b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.911602 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d123-abc7-4c13-97fa-e524312c44b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.911885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d123-abc7-4c13-97fa-e524312c44b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.912668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11d123-abc7-4c13-97fa-e524312c44b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:14.914266 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11d123-abc7-4c13-97fa-e524312c44b9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:14.925394 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1406/2815] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:14.931256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e6aec-e406-4e40-9412-0bc5ca95f73b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:14.946227 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e6aec-e406-4e40-9412-0bc5ca95f73b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:14.953246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6418476-d8e0-4186-a315-8c7a37ac9708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:14.954161 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6418476-d8e0-4186-a315-8c7a37ac9708 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:14.954434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6418476-d8e0-4186-a315-8c7a37ac9708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:14.954703 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6418476-d8e0-4186-a315-8c7a37ac9708 admin admin] PATH_INFO: `/v3/projects/5ee098e49ac4471cafc34cd5d01f8b6c/users/0b43961bf5054e1ba1ce44b61929bfc0/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:14.972029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6418476-d8e0-4186-a315-8c7a37ac9708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee098e49ac4471cafc34cd5d01f8b6c, user_id=0b43961bf5054e1ba1ce44b61929bfc0, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:14.974701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6418476-d8e0-4186-a315-8c7a37ac9708 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:14.990507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1410/2816] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:14 2022] PUT /identity/v3/projects/5ee098e49ac4471cafc34cd5d01f8b6c/users/0b43961bf5054e1ba1ce44b61929bfc0/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:14.994816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11d123-abc7-4c13-97fa-e524312c44b9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:15.005434 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11d123-abc7-4c13-97fa-e524312c44b9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:15.025170 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:15.064629 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2cdea-ba28-422a-8cd1-20f6c0a886eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:15.065296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2cdea-ba28-422a-8cd1-20f6c0a886eb admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:15.065579 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2cdea-ba28-422a-8cd1-20f6c0a886eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:15.065847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2cdea-ba28-422a-8cd1-20f6c0a886eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:15.066580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2cdea-ba28-422a-8cd1-20f6c0a886eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:15.068218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2cdea-ba28-422a-8cd1-20f6c0a886eb admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:15.078048 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1407/2817] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:15.082801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6418476-d8e0-4186-a315-8c7a37ac9708 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:15.088818 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:15.095961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6418476-d8e0-4186-a315-8c7a37ac9708 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:15.101389 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3f4b6-d25d-4b99-b4ec-32d1b2d472f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:15.102287 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3f4b6-d25d-4b99-b4ec-32d1b2d472f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:15.102581 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3f4b6-d25d-4b99-b4ec-32d1b2d472f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:15.102859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3f4b6-d25d-4b99-b4ec-32d1b2d472f5 admin admin] PATH_INFO: `/v3/projects/5ee098e49ac4471cafc34cd5d01f8b6c/users/0b43961bf5054e1ba1ce44b61929bfc0/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:15.104959 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3f4b6-d25d-4b99-b4ec-32d1b2d472f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee098e49ac4471cafc34cd5d01f8b6c, user_id=0b43961bf5054e1ba1ce44b61929bfc0, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:15.108198 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3f4b6-d25d-4b99-b4ec-32d1b2d472f5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:15.118997 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1411/2818] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:15 2022] PUT /identity/v3/projects/5ee098e49ac4471cafc34cd5d01f8b6c/users/0b43961bf5054e1ba1ce44b61929bfc0/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:15.130276 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1310fac-16b7-4ddf-9b94-26e2d2f09044 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:15.130645 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1310fac-16b7-4ddf-9b94-26e2d2f09044 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:15.131039 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1310fac-16b7-4ddf-9b94-26e2d2f09044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:15.165809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c1310fac-16b7-4ddf-9b94-26e2d2f09044 None None] MFA Rules not processed for user `0b43961bf5054e1ba1ce44b61929bfc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:15.167014 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c1310fac-16b7-4ddf-9b94-26e2d2f09044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:15.243473 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1408/2819] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:15:15.259064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3f4b6-d25d-4b99-b4ec-32d1b2d472f5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:15.261173 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-94b3f4b6-d25d-4b99-b4ec-32d1b2d472f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:15.335252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3f4b6-d25d-4b99-b4ec-32d1b2d472f5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:15.342966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189853fd-c15a-4920-823a-e1e78e111e3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:15.343612 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189853fd-c15a-4920-823a-e1e78e111e3f service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:15.343897 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189853fd-c15a-4920-823a-e1e78e111e3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:15.344268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189853fd-c15a-4920-823a-e1e78e111e3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:15.353981 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189853fd-c15a-4920-823a-e1e78e111e3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:15.355432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189853fd-c15a-4920-823a-e1e78e111e3f service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:15.365629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1412/2820] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:15.425388 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1310fac-16b7-4ddf-9b94-26e2d2f09044 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:15.427167 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c1310fac-16b7-4ddf-9b94-26e2d2f09044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:15.441072 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1310fac-16b7-4ddf-9b94-26e2d2f09044 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:15.445695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ef14b-ac7f-40d7-a110-2033894d88e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:15.446339 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ef14b-ac7f-40d7-a110-2033894d88e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:15.446615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ef14b-ac7f-40d7-a110-2033894d88e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:15.446885 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ef14b-ac7f-40d7-a110-2033894d88e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:15.512201 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ef14b-ac7f-40d7-a110-2033894d88e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:15.513821 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ef14b-ac7f-40d7-a110-2033894d88e8 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:15.523067 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1409/2821] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:18.875806 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189853fd-c15a-4920-823a-e1e78e111e3f service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:18.939292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189853fd-c15a-4920-823a-e1e78e111e3f service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:18.944371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:18.945165 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:18.945446 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:18.945714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] PATH_INFO: `/v3/users/d3f0e3a15c3b47cebd56b083b296fef2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:18.946934 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3f0e3a15c3b47cebd56b083b296fef2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:18.948881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:19.001024 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3f0e3a15c3b47cebd56b083b296fef2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.008333 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3f0e3a15c3b47cebd56b083b296fef2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.009013 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3f0e3a15c3b47cebd56b083b296fef2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.018271 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3f0e3a15c3b47cebd56b083b296fef2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.030545 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3f0e3a15c3b47cebd56b083b296fef2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.036433 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1413/2822] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:15:18 2022] DELETE /identity/v3/users/d3f0e3a15c3b47cebd56b083b296fef2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:19.041053 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5965a0f-618a-4a1b-b5ae-3ab90f6eb4e5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:19.041555 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5965a0f-618a-4a1b-b5ae-3ab90f6eb4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:19.041667 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5965a0f-618a-4a1b-b5ae-3ab90f6eb4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:19.059682 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a5965a0f-618a-4a1b-b5ae-3ab90f6eb4e5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:19.060993 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a5965a0f-618a-4a1b-b5ae-3ab90f6eb4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:19.105711 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:19.124813 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1410/2823] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:19.135006 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:19.136745 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:19.151817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:19.198126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b08938-79ee-4f83-87ff-d718902c8d25 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:19.201671 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b86ba4-f9a7-4f17-a7e7-e5db0385f175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:19.202315 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b86ba4-f9a7-4f17-a7e7-e5db0385f175 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:19.202594 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b86ba4-f9a7-4f17-a7e7-e5db0385f175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:19.202865 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b86ba4-f9a7-4f17-a7e7-e5db0385f175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:19.211821 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b86ba4-f9a7-4f17-a7e7-e5db0385f175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:19.213310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b86ba4-f9a7-4f17-a7e7-e5db0385f175 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:19.223420 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1414/2824] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:19.513300 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5965a0f-618a-4a1b-b5ae-3ab90f6eb4e5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:19.514803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a5965a0f-618a-4a1b-b5ae-3ab90f6eb4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:19.527538 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5965a0f-618a-4a1b-b5ae-3ab90f6eb4e5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:19.532875 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:19.533881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:19.534175 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:19.534441 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] PATH_INFO: `/v3/projects/84964bfccd4a4620babfe3e36066cc81` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:19.535794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=84964bfccd4a4620babfe3e36066cc81)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:19.537751 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:19.596768 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84964bfccd4a4620babfe3e36066cc81', 'request_id': 'req-1cdfb309-6852-4b4f-a699-cfb9960776fe'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.602823 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84964bfccd4a4620babfe3e36066cc81', 'request_id': 'req-1cdfb309-6852-4b4f-a699-cfb9960776fe'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.613219 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84964bfccd4a4620babfe3e36066cc81', 'request_id': 'req-1cdfb309-6852-4b4f-a699-cfb9960776fe'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.623297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] The token cache is being invalidate because project 84964bfccd4a4620babfe3e36066cc81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:15:19.623580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.624214 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.625750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1411/2825] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:15:19 2022] DELETE /identity/v3/projects/84964bfccd4a4620babfe3e36066cc81 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:15:19.630022 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b86ba4-f9a7-4f17-a7e7-e5db0385f175 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:19.639956 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b86ba4-f9a7-4f17-a7e7-e5db0385f175 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:19.697725 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:19.698604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:19.698881 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:19.699157 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] PATH_INFO: `/v3/users/0b43961bf5054e1ba1ce44b61929bfc0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:19.700377 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b43961bf5054e1ba1ce44b61929bfc0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:19.702303 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:19.748806 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b43961bf5054e1ba1ce44b61929bfc0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.755498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b43961bf5054e1ba1ce44b61929bfc0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.756237 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b43961bf5054e1ba1ce44b61929bfc0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.764449 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b43961bf5054e1ba1ce44b61929bfc0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.776520 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e2d1bed9-f806-4d3e-82aa-b2325c543cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b43961bf5054e1ba1ce44b61929bfc0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:19.782469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1415/2826] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:15:19 2022] DELETE /identity/v3/users/0b43961bf5054e1ba1ce44b61929bfc0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:20.070381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:20.071852 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:20.089464 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfb309-6852-4b4f-a699-cfb9960776fe admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:20.151743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:20.152974 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.153490 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.154306 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] PATH_INFO: `/v3/projects/5ee098e49ac4471cafc34cd5d01f8b6c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.156450 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ee098e49ac4471cafc34cd5d01f8b6c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:20.158294 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:20.211052 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ee098e49ac4471cafc34cd5d01f8b6c', 'request_id': 'req-e94cab61-0ef3-4440-808c-c02d23dc9bd8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:20.217337 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ee098e49ac4471cafc34cd5d01f8b6c', 'request_id': 'req-e94cab61-0ef3-4440-808c-c02d23dc9bd8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:20.226202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ee098e49ac4471cafc34cd5d01f8b6c', 'request_id': 'req-e94cab61-0ef3-4440-808c-c02d23dc9bd8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:20.232688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:20.237868 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] The token cache is being invalidate because project 5ee098e49ac4471cafc34cd5d01f8b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:15:20.238161 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:20.239010 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e94cab61-0ef3-4440-808c-c02d23dc9bd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:20.240969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1412/2827] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:15:20 2022] DELETE /identity/v3/projects/5ee098e49ac4471cafc34cd5d01f8b6c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:20.246423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3577465-4904-4dc8-9839-ddb2ce0ce25e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.246961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3577465-4904-4dc8-9839-ddb2ce0ce25e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.246961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3577465-4904-4dc8-9839-ddb2ce0ce25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.264948 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a3577465-4904-4dc8-9839-ddb2ce0ce25e None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:20.266091 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a3577465-4904-4dc8-9839-ddb2ce0ce25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:20.296213 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:20.329702 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1416/2828] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:20.337566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abfaef-0a5b-4634-9d9f-86934556f309 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.337853 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abfaef-0a5b-4634-9d9f-86934556f309 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.338143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abfaef-0a5b-4634-9d9f-86934556f309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.358822 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-40abfaef-0a5b-4634-9d9f-86934556f309 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:20.360024 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-40abfaef-0a5b-4634-9d9f-86934556f309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:20.368689 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1413/2829] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:20.373104 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3577465-4904-4dc8-9839-ddb2ce0ce25e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:20.383942 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3577465-4904-4dc8-9839-ddb2ce0ce25e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:20.390531 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d22524-686d-4071-a4b3-0fc59f8fd92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:20.391188 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d22524-686d-4071-a4b3-0fc59f8fd92c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.391476 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d22524-686d-4071-a4b3-0fc59f8fd92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.391746 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d22524-686d-4071-a4b3-0fc59f8fd92c admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.392523 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d22524-686d-4071-a4b3-0fc59f8fd92c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:15:20.392836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d22524-686d-4071-a4b3-0fc59f8fd92c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:20.394551 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d22524-686d-4071-a4b3-0fc59f8fd92c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:20.406850 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1417/2830] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:15:20.411120 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abfaef-0a5b-4634-9d9f-86934556f309 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:20.421295 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abfaef-0a5b-4634-9d9f-86934556f309 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:20.427226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254888fd-5a12-4a14-8ff0-bc00933098a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:20.427869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254888fd-5a12-4a14-8ff0-bc00933098a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.428163 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254888fd-5a12-4a14-8ff0-bc00933098a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.428439 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254888fd-5a12-4a14-8ff0-bc00933098a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.429136 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254888fd-5a12-4a14-8ff0-bc00933098a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:20.431061 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254888fd-5a12-4a14-8ff0-bc00933098a5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:20.459957 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1414/2831] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:15:20 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:20.464837 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d22524-686d-4071-a4b3-0fc59f8fd92c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:20.478090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d22524-686d-4071-a4b3-0fc59f8fd92c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:20.554900 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab81fa-b4c5-426f-a044-7d826f7a3ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:20.555477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab81fa-b4c5-426f-a044-7d826f7a3ddd admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.555873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab81fa-b4c5-426f-a044-7d826f7a3ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.556432 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab81fa-b4c5-426f-a044-7d826f7a3ddd admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.557507 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab81fa-b4c5-426f-a044-7d826f7a3ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:20.559499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab81fa-b4c5-426f-a044-7d826f7a3ddd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:20.583979 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1418/2832] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:15:20 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:20.588381 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254888fd-5a12-4a14-8ff0-bc00933098a5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:20.598924 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254888fd-5a12-4a14-8ff0-bc00933098a5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:20.603655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca8e24-7f6b-4cab-8597-059e3eb7b299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:20.604372 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca8e24-7f6b-4cab-8597-059e3eb7b299 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.604650 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca8e24-7f6b-4cab-8597-059e3eb7b299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.604922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca8e24-7f6b-4cab-8597-059e3eb7b299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.605648 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca8e24-7f6b-4cab-8597-059e3eb7b299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:20.607245 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca8e24-7f6b-4cab-8597-059e3eb7b299 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:20.617640 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1415/2833] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:20.621636 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab81fa-b4c5-426f-a044-7d826f7a3ddd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:20.632894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab81fa-b4c5-426f-a044-7d826f7a3ddd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:20.638373 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2fa26-07e2-47a0-a2ca-189d8019631d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:20.639042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2fa26-07e2-47a0-a2ca-189d8019631d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.639323 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2fa26-07e2-47a0-a2ca-189d8019631d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.639599 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2fa26-07e2-47a0-a2ca-189d8019631d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.640369 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2fa26-07e2-47a0-a2ca-189d8019631d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:20.641989 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2fa26-07e2-47a0-a2ca-189d8019631d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:20.652859 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1419/2834] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:20.656983 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca8e24-7f6b-4cab-8597-059e3eb7b299 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:20.667385 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca8e24-7f6b-4cab-8597-059e3eb7b299 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:20.673102 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3e404-3ab0-4b84-a32d-981f592ebe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:20.673987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3e404-3ab0-4b84-a32d-981f592ebe05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.674322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3e404-3ab0-4b84-a32d-981f592ebe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.674694 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3e404-3ab0-4b84-a32d-981f592ebe05 admin admin] PATH_INFO: `/v3/projects/7b4c0fdd1c174253b0983ff3bb79cad0/users/83f9a6bdadc34aa7876e439719f1eda7/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.690032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3e404-3ab0-4b84-a32d-981f592ebe05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4c0fdd1c174253b0983ff3bb79cad0, user_id=83f9a6bdadc34aa7876e439719f1eda7, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:20.692729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3e404-3ab0-4b84-a32d-981f592ebe05 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:20.706475 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1416/2835] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:20 2022] PUT /identity/v3/projects/7b4c0fdd1c174253b0983ff3bb79cad0/users/83f9a6bdadc34aa7876e439719f1eda7/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:20.711633 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66737a-a5ec-472f-96bb-577002ad9e1b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.711930 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66737a-a5ec-472f-96bb-577002ad9e1b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.712249 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66737a-a5ec-472f-96bb-577002ad9e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.731787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9c66737a-a5ec-472f-96bb-577002ad9e1b None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:20.733049 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9c66737a-a5ec-472f-96bb-577002ad9e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:20.808627 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1420/2836] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:20.818824 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3e404-3ab0-4b84-a32d-981f592ebe05 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:20.820894 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6eb3e404-3ab0-4b84-a32d-981f592ebe05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:20.885341 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3e404-3ab0-4b84-a32d-981f592ebe05 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:20.890284 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252e98d-d94c-4373-927d-bd3b56d4a30e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:20.890977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252e98d-d94c-4373-927d-bd3b56d4a30e service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:20.891260 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252e98d-d94c-4373-927d-bd3b56d4a30e service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:20.891540 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252e98d-d94c-4373-927d-bd3b56d4a30e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:20.903818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252e98d-d94c-4373-927d-bd3b56d4a30e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:20.905307 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252e98d-d94c-4373-927d-bd3b56d4a30e service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:20.916146 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1417/2837] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:21.548656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f0f2b-19da-4716-90f5-e97bdb2d38c0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:21.549273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f0f2b-19da-4716-90f5-e97bdb2d38c0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:21.549273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f0f2b-19da-4716-90f5-e97bdb2d38c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:21.576982 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ee3f0f2b-19da-4716-90f5-e97bdb2d38c0 None None] MFA Rules not processed for user `83f9a6bdadc34aa7876e439719f1eda7`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:21.578259 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ee3f0f2b-19da-4716-90f5-e97bdb2d38c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:21.611292 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:21.647632 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1421/2838] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:21.652226 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252e98d-d94c-4373-927d-bd3b56d4a30e service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:21.662930 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252e98d-d94c-4373-927d-bd3b56d4a30e service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:21.668621 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a44069-7ad9-42ad-b5ae-841266e8ae40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:21.669293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a44069-7ad9-42ad-b5ae-841266e8ae40 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:21.669589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a44069-7ad9-42ad-b5ae-841266e8ae40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:21.669812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a44069-7ad9-42ad-b5ae-841266e8ae40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:21.671036 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a44069-7ad9-42ad-b5ae-841266e8ae40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:21.673242 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a44069-7ad9-42ad-b5ae-841266e8ae40 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:21.699408 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1418/2839] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:15:21 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:21.703462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f0f2b-19da-4716-90f5-e97bdb2d38c0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:21.714780 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f0f2b-19da-4716-90f5-e97bdb2d38c0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:21.771738 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f7b1d-aa4b-4013-b7e9-edb5a7ab4334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:21.772448 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f7b1d-aa4b-4013-b7e9-edb5a7ab4334 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:21.772730 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f7b1d-aa4b-4013-b7e9-edb5a7ab4334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:21.773005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f7b1d-aa4b-4013-b7e9-edb5a7ab4334 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:21.773690 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f7b1d-aa4b-4013-b7e9-edb5a7ab4334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:21.775618 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f7b1d-aa4b-4013-b7e9-edb5a7ab4334 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:21.797065 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1422/2840] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:15:21 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:21.801969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a44069-7ad9-42ad-b5ae-841266e8ae40 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:21.812535 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a44069-7ad9-42ad-b5ae-841266e8ae40 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:21.818631 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3103f99-ece3-4955-9175-65f4793aba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:21.819358 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3103f99-ece3-4955-9175-65f4793aba67 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:21.819617 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3103f99-ece3-4955-9175-65f4793aba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:21.819887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3103f99-ece3-4955-9175-65f4793aba67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:21.820657 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3103f99-ece3-4955-9175-65f4793aba67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:21.822271 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3103f99-ece3-4955-9175-65f4793aba67 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:21.834478 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1419/2841] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:21.838527 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f7b1d-aa4b-4013-b7e9-edb5a7ab4334 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:21.849005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f7b1d-aa4b-4013-b7e9-edb5a7ab4334 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:21.854866 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f0f4a-b749-4ffa-9255-adab18e6034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:21.855766 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f0f4a-b749-4ffa-9255-adab18e6034d admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:21.856035 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f0f4a-b749-4ffa-9255-adab18e6034d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:21.856344 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f0f4a-b749-4ffa-9255-adab18e6034d admin admin] PATH_INFO: `/v3/projects/0d386663d1e34a00b2e014e390de976a/users/d46ae8af3c0d43b1ae265b21ae4a0ff7/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:21.871936 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f0f4a-b749-4ffa-9255-adab18e6034d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d386663d1e34a00b2e014e390de976a, user_id=d46ae8af3c0d43b1ae265b21ae4a0ff7, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:21.874633 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f0f4a-b749-4ffa-9255-adab18e6034d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:21.888558 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1423/2842] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:21 2022] PUT /identity/v3/projects/0d386663d1e34a00b2e014e390de976a/users/d46ae8af3c0d43b1ae265b21ae4a0ff7/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:21.892524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3103f99-ece3-4955-9175-65f4793aba67 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:21.903285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3103f99-ece3-4955-9175-65f4793aba67 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:21.972506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215b385-57f6-47e6-8627-fcfba3c08d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:21.973097 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215b385-57f6-47e6-8627-fcfba3c08d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:21.973378 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215b385-57f6-47e6-8627-fcfba3c08d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:21.973647 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215b385-57f6-47e6-8627-fcfba3c08d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:21.974743 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215b385-57f6-47e6-8627-fcfba3c08d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:21.977204 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215b385-57f6-47e6-8627-fcfba3c08d4d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:21.985494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:21.992641 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1420/2843] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:21.996653 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f0f4a-b749-4ffa-9255-adab18e6034d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:22.008000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f0f4a-b749-4ffa-9255-adab18e6034d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:22.012899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493aa48-626f-45d6-828d-bf50f7aa90d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:22.013796 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493aa48-626f-45d6-828d-bf50f7aa90d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:22.014073 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493aa48-626f-45d6-828d-bf50f7aa90d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:22.014354 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493aa48-626f-45d6-828d-bf50f7aa90d8 admin admin] PATH_INFO: `/v3/projects/0d386663d1e34a00b2e014e390de976a/users/d46ae8af3c0d43b1ae265b21ae4a0ff7/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:22.016469 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493aa48-626f-45d6-828d-bf50f7aa90d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d386663d1e34a00b2e014e390de976a, user_id=d46ae8af3c0d43b1ae265b21ae4a0ff7, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:22.019179 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493aa48-626f-45d6-828d-bf50f7aa90d8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:22.029696 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1424/2844] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:21 2022] PUT /identity/v3/projects/0d386663d1e34a00b2e014e390de976a/users/d46ae8af3c0d43b1ae265b21ae4a0ff7/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:22.678987 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddd8e7-482c-4b3b-a637-ecb46b06bd43 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:22.679581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddd8e7-482c-4b3b-a637-ecb46b06bd43 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:22.679581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddd8e7-482c-4b3b-a637-ecb46b06bd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:22.706806 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-02ddd8e7-482c-4b3b-a637-ecb46b06bd43 None None] MFA Rules not processed for user `d46ae8af3c0d43b1ae265b21ae4a0ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:22.708023 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-02ddd8e7-482c-4b3b-a637-ecb46b06bd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:22.785330 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1421/2845] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:22.797310 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493aa48-626f-45d6-828d-bf50f7aa90d8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:22.799168 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c493aa48-626f-45d6-828d-bf50f7aa90d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:22.861907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493aa48-626f-45d6-828d-bf50f7aa90d8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:22.865582 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e225669-2c7b-44b7-a4d5-cf8045e1d8bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:22.866240 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e225669-2c7b-44b7-a4d5-cf8045e1d8bb service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:22.866518 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e225669-2c7b-44b7-a4d5-cf8045e1d8bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:22.866787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e225669-2c7b-44b7-a4d5-cf8045e1d8bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:22.899645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:22.924505 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e225669-2c7b-44b7-a4d5-cf8045e1d8bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:22.925985 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e225669-2c7b-44b7-a4d5-cf8045e1d8bb service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:22.935047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1425/2846] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:24.310589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddd8e7-482c-4b3b-a637-ecb46b06bd43 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:24.312288 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-02ddd8e7-482c-4b3b-a637-ecb46b06bd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:24.326152 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddd8e7-482c-4b3b-a637-ecb46b06bd43 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:24.331051 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95e822-e2b7-4728-9a75-91d9733f4cfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:24.331718 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95e822-e2b7-4728-9a75-91d9733f4cfb service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:24.331992 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95e822-e2b7-4728-9a75-91d9733f4cfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:24.332302 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95e822-e2b7-4728-9a75-91d9733f4cfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:24.342506 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95e822-e2b7-4728-9a75-91d9733f4cfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:24.343955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95e822-e2b7-4728-9a75-91d9733f4cfb service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:24.353783 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1422/2847] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:32.931155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=11 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:34.349569 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=40 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:37.577698 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e225669-2c7b-44b7-a4d5-cf8045e1d8bb service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:37.698858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e225669-2c7b-44b7-a4d5-cf8045e1d8bb service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:37.704863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:37.705733 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:37.706033 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:37.706301 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] PATH_INFO: `/v3/users/83f9a6bdadc34aa7876e439719f1eda7` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:37.707569 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f9a6bdadc34aa7876e439719f1eda7)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:37.709525 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:37.761997 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f9a6bdadc34aa7876e439719f1eda7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:37.768848 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f9a6bdadc34aa7876e439719f1eda7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:37.769494 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f9a6bdadc34aa7876e439719f1eda7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:37.778021 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f9a6bdadc34aa7876e439719f1eda7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:37.790155 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f9a6bdadc34aa7876e439719f1eda7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:37.797563 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1426/2848] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:15:37 2022] DELETE /identity/v3/users/83f9a6bdadc34aa7876e439719f1eda7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:37.973206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95e822-e2b7-4728-9a75-91d9733f4cfb service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:38.014826 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95e822-e2b7-4728-9a75-91d9733f4cfb service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:38.075722 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:38.076653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:38.076932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:38.077206 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] PATH_INFO: `/v3/projects/7b4c0fdd1c174253b0983ff3bb79cad0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:38.078687 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4c0fdd1c174253b0983ff3bb79cad0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:38.080653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:38.128035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4c0fdd1c174253b0983ff3bb79cad0', 'request_id': 'req-80b35551-c88d-49c2-ac19-1526da560797'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.134233 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4c0fdd1c174253b0983ff3bb79cad0', 'request_id': 'req-80b35551-c88d-49c2-ac19-1526da560797'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.144126 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4c0fdd1c174253b0983ff3bb79cad0', 'request_id': 'req-80b35551-c88d-49c2-ac19-1526da560797'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.169232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] The token cache is being invalidate because project 7b4c0fdd1c174253b0983ff3bb79cad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:15:38.169521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.170166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.172338 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1423/2849] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:15:37 2022] DELETE /identity/v3/projects/7b4c0fdd1c174253b0983ff3bb79cad0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:38.176675 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:38.178174 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:38.201892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f632e38-7040-42b2-a84e-6189c3305059 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:38.227863 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:38.260363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:38.261253 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:38.261529 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:38.261801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] PATH_INFO: `/v3/users/d46ae8af3c0d43b1ae265b21ae4a0ff7` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:38.263028 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46ae8af3c0d43b1ae265b21ae4a0ff7)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:38.265042 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:38.320707 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46ae8af3c0d43b1ae265b21ae4a0ff7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.327404 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46ae8af3c0d43b1ae265b21ae4a0ff7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.328183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46ae8af3c0d43b1ae265b21ae4a0ff7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.336924 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46ae8af3c0d43b1ae265b21ae4a0ff7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.354304 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5c979af1-ce66-4d9a-b7a9-58f8a0971593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46ae8af3c0d43b1ae265b21ae4a0ff7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.362317 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1427/2850] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:15:38 2022] DELETE /identity/v3/users/d46ae8af3c0d43b1ae265b21ae4a0ff7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:38.548977 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:38.550437 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:38.562782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b35551-c88d-49c2-ac19-1526da560797 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:38.612695 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:38.620348 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:38.621270 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:38.621554 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:38.621818 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] PATH_INFO: `/v3/projects/0d386663d1e34a00b2e014e390de976a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:38.623144 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d386663d1e34a00b2e014e390de976a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:38.625322 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:38.673721 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d386663d1e34a00b2e014e390de976a', 'request_id': 'req-06fbb67d-63e6-43af-8247-6ac7a553af88'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.679688 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d386663d1e34a00b2e014e390de976a', 'request_id': 'req-06fbb67d-63e6-43af-8247-6ac7a553af88'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.689796 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d386663d1e34a00b2e014e390de976a', 'request_id': 'req-06fbb67d-63e6-43af-8247-6ac7a553af88'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.700597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] The token cache is being invalidate because project 0d386663d1e34a00b2e014e390de976a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:15:38.700887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.701557 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06fbb67d-63e6-43af-8247-6ac7a553af88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:38.703290 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1424/2851] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:15:38 2022] DELETE /identity/v3/projects/0d386663d1e34a00b2e014e390de976a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:38.708642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7e81e-02be-4565-b53c-44ffe23fb147 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:38.709170 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7e81e-02be-4565-b53c-44ffe23fb147 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:38.709289 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7e81e-02be-4565-b53c-44ffe23fb147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:38.776604 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-25b7e81e-02be-4565-b53c-44ffe23fb147 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:38.777716 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-25b7e81e-02be-4565-b53c-44ffe23fb147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:38.840969 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1428/2852] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 21:15:38.849766 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232982b3-67f0-40bb-8e6b-87860c3733a2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:38.850084 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232982b3-67f0-40bb-8e6b-87860c3733a2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:38.850384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232982b3-67f0-40bb-8e6b-87860c3733a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:38.869191 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-232982b3-67f0-40bb-8e6b-87860c3733a2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:38.870401 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-232982b3-67f0-40bb-8e6b-87860c3733a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:38.879774 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1425/2853] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:38.884201 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7e81e-02be-4565-b53c-44ffe23fb147 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:38.894142 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7e81e-02be-4565-b53c-44ffe23fb147 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:38.899792 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a673cd-7b46-46fa-b94e-ed88a5a58ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:38.900499 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a673cd-7b46-46fa-b94e-ed88a5a58ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:38.900791 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a673cd-7b46-46fa-b94e-ed88a5a58ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:38.901055 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a673cd-7b46-46fa-b94e-ed88a5a58ee0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:38.901787 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a673cd-7b46-46fa-b94e-ed88a5a58ee0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:15:38.902090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a673cd-7b46-46fa-b94e-ed88a5a58ee0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:38.903713 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a673cd-7b46-46fa-b94e-ed88a5a58ee0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:38.916254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1429/2854] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:15:38.921037 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232982b3-67f0-40bb-8e6b-87860c3733a2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:38.931013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232982b3-67f0-40bb-8e6b-87860c3733a2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:38.937275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835bd05-6fdd-42d1-953a-9ff0e370db22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:38.937887 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835bd05-6fdd-42d1-953a-9ff0e370db22 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:38.938166 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835bd05-6fdd-42d1-953a-9ff0e370db22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:38.938431 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835bd05-6fdd-42d1-953a-9ff0e370db22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:38.939125 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835bd05-6fdd-42d1-953a-9ff0e370db22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:38.941066 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835bd05-6fdd-42d1-953a-9ff0e370db22 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:38.969833 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1426/2855] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:15:38 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:38.974150 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a673cd-7b46-46fa-b94e-ed88a5a58ee0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:38.980621 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:38.984676 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a673cd-7b46-46fa-b94e-ed88a5a58ee0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.047762 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b92141-da87-47d9-b543-94d649f7d37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.048624 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b92141-da87-47d9-b543-94d649f7d37f admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.048998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b92141-da87-47d9-b543-94d649f7d37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.049281 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b92141-da87-47d9-b543-94d649f7d37f admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.050349 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b92141-da87-47d9-b543-94d649f7d37f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:39.053200 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b92141-da87-47d9-b543-94d649f7d37f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:39.079492 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1430/2856] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:15:38 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:39.084149 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835bd05-6fdd-42d1-953a-9ff0e370db22 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:39.095384 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835bd05-6fdd-42d1-953a-9ff0e370db22 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.101914 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05342-e31f-44b3-aeb2-02318ff04670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.102591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05342-e31f-44b3-aeb2-02318ff04670 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.102873 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05342-e31f-44b3-aeb2-02318ff04670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.103145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05342-e31f-44b3-aeb2-02318ff04670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.103877 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd05342-e31f-44b3-aeb2-02318ff04670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:39.105518 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd05342-e31f-44b3-aeb2-02318ff04670 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:39.116837 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1427/2857] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:39.121090 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b92141-da87-47d9-b543-94d649f7d37f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:39.131408 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b92141-da87-47d9-b543-94d649f7d37f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.136915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8dcbe-9fc6-48aa-8b95-53a395c9792f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.137555 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8dcbe-9fc6-48aa-8b95-53a395c9792f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.137828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8dcbe-9fc6-48aa-8b95-53a395c9792f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.138099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8dcbe-9fc6-48aa-8b95-53a395c9792f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.138824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8dcbe-9fc6-48aa-8b95-53a395c9792f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:39.140506 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8dcbe-9fc6-48aa-8b95-53a395c9792f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:39.151030 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1431/2858] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:15:39.155179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05342-e31f-44b3-aeb2-02318ff04670 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:39.165819 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05342-e31f-44b3-aeb2-02318ff04670 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.171801 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9122ba0-4efc-4bc7-9a2f-4f2107d8d5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.172730 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9122ba0-4efc-4bc7-9a2f-4f2107d8d5e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.173004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9122ba0-4efc-4bc7-9a2f-4f2107d8d5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.173296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9122ba0-4efc-4bc7-9a2f-4f2107d8d5e7 admin admin] PATH_INFO: `/v3/projects/c6943bf93cda4888bad2dfd664681d0c/users/6cc7cd21122b4dae92178044851ae907/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.189382 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9122ba0-4efc-4bc7-9a2f-4f2107d8d5e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6943bf93cda4888bad2dfd664681d0c, user_id=6cc7cd21122b4dae92178044851ae907, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:39.192013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9122ba0-4efc-4bc7-9a2f-4f2107d8d5e7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:39.207253 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1428/2859] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:39 2022] PUT /identity/v3/projects/c6943bf93cda4888bad2dfd664681d0c/users/6cc7cd21122b4dae92178044851ae907/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:39.217740 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5c7c5-0a65-4ac1-9b00-6f321b59fdb2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.218029 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5c7c5-0a65-4ac1-9b00-6f321b59fdb2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.218307 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5c7c5-0a65-4ac1-9b00-6f321b59fdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.244628 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c1e5c7c5-0a65-4ac1-9b00-6f321b59fdb2 None None] MFA Rules not processed for user `6cc7cd21122b4dae92178044851ae907`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:39.245824 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c1e5c7c5-0a65-4ac1-9b00-6f321b59fdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:39.312018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1432/2860] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:39.316863 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9122ba0-4efc-4bc7-9a2f-4f2107d8d5e7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:39.381268 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9122ba0-4efc-4bc7-9a2f-4f2107d8d5e7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.386661 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67845720-e89e-470f-aa95-42bc94e2c564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.387313 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67845720-e89e-470f-aa95-42bc94e2c564 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.387588 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67845720-e89e-470f-aa95-42bc94e2c564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.387861 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67845720-e89e-470f-aa95-42bc94e2c564 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.388574 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67845720-e89e-470f-aa95-42bc94e2c564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:39.390567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67845720-e89e-470f-aa95-42bc94e2c564 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:39.419238 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1429/2861] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:15:39 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:39.423966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5c7c5-0a65-4ac1-9b00-6f321b59fdb2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:39.435025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5c7c5-0a65-4ac1-9b00-6f321b59fdb2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.492252 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4021fd-c5f5-4315-b491-6e9e29fb62b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.492944 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4021fd-c5f5-4315-b491-6e9e29fb62b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.493220 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4021fd-c5f5-4315-b491-6e9e29fb62b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.493480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4021fd-c5f5-4315-b491-6e9e29fb62b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.494194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4021fd-c5f5-4315-b491-6e9e29fb62b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:39.496138 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4021fd-c5f5-4315-b491-6e9e29fb62b8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:39.516683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1433/2862] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:15:39 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:39.520771 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67845720-e89e-470f-aa95-42bc94e2c564 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:39.531187 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67845720-e89e-470f-aa95-42bc94e2c564 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.536512 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae909434-001e-4810-a1d6-df1192e8984e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.537177 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae909434-001e-4810-a1d6-df1192e8984e admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.537463 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae909434-001e-4810-a1d6-df1192e8984e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.537737 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae909434-001e-4810-a1d6-df1192e8984e admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.538508 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae909434-001e-4810-a1d6-df1192e8984e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:39.540145 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae909434-001e-4810-a1d6-df1192e8984e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:39.541393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:39.554143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1430/2863] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:39.557974 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4021fd-c5f5-4315-b491-6e9e29fb62b8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:39.563831 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:39.570351 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4021fd-c5f5-4315-b491-6e9e29fb62b8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.576441 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbad18f-2138-4035-9fbb-70295389d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.577336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbad18f-2138-4035-9fbb-70295389d083 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.577620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbad18f-2138-4035-9fbb-70295389d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.577895 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbad18f-2138-4035-9fbb-70295389d083 admin admin] PATH_INFO: `/v3/projects/f3a461ea782148d49f6b3afded195920/users/6dbbc86a10014ad49aef4d0971c09c32/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.594353 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbad18f-2138-4035-9fbb-70295389d083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a461ea782148d49f6b3afded195920, user_id=6dbbc86a10014ad49aef4d0971c09c32, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:39.597152 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbad18f-2138-4035-9fbb-70295389d083 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:39.612843 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1434/2864] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:39 2022] PUT /identity/v3/projects/f3a461ea782148d49f6b3afded195920/users/6dbbc86a10014ad49aef4d0971c09c32/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:39.617202 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae909434-001e-4810-a1d6-df1192e8984e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:39.627613 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae909434-001e-4810-a1d6-df1192e8984e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.690121 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4a893-ec4f-4833-be95-c4138ecee81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.690787 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4a893-ec4f-4833-be95-c4138ecee81c admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.691176 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4a893-ec4f-4833-be95-c4138ecee81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.691458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4a893-ec4f-4833-be95-c4138ecee81c admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.692224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4a893-ec4f-4833-be95-c4138ecee81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:39.693816 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4a893-ec4f-4833-be95-c4138ecee81c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:39.704224 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1431/2865] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:39.708736 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbad18f-2138-4035-9fbb-70295389d083 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:39.719858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbad18f-2138-4035-9fbb-70295389d083 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.724757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28dc639-aa47-4205-9ceb-e9c7b24f0bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.725643 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28dc639-aa47-4205-9ceb-e9c7b24f0bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.725917 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28dc639-aa47-4205-9ceb-e9c7b24f0bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.726187 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28dc639-aa47-4205-9ceb-e9c7b24f0bf4 admin admin] PATH_INFO: `/v3/projects/f3a461ea782148d49f6b3afded195920/users/6dbbc86a10014ad49aef4d0971c09c32/roles/60bd1c44ce154338a7face8949d07281` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.728620 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28dc639-aa47-4205-9ceb-e9c7b24f0bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a461ea782148d49f6b3afded195920, user_id=6dbbc86a10014ad49aef4d0971c09c32, role_id=60bd1c44ce154338a7face8949d07281)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:39.731262 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28dc639-aa47-4205-9ceb-e9c7b24f0bf4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:39.741606 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1435/2866] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:39 2022] PUT /identity/v3/projects/f3a461ea782148d49f6b3afded195920/users/6dbbc86a10014ad49aef4d0971c09c32/roles/60bd1c44ce154338a7face8949d07281 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:39.753065 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957365a9-4989-436d-afaf-0c2717153971 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.753353 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957365a9-4989-436d-afaf-0c2717153971 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.753655 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957365a9-4989-436d-afaf-0c2717153971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:39.781154 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-957365a9-4989-436d-afaf-0c2717153971 None None] MFA Rules not processed for user `6dbbc86a10014ad49aef4d0971c09c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:39.782345 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-957365a9-4989-436d-afaf-0c2717153971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:39.853135 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1432/2867] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:39.864548 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28dc639-aa47-4205-9ceb-e9c7b24f0bf4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:39.866608 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a28dc639-aa47-4205-9ceb-e9c7b24f0bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:39.945645 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28dc639-aa47-4205-9ceb-e9c7b24f0bf4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:39.949623 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3de96c-a34a-4096-9769-b1b9e4da5100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:39.950268 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3de96c-a34a-4096-9769-b1b9e4da5100 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:39.950694 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3de96c-a34a-4096-9769-b1b9e4da5100 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:39.950970 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3de96c-a34a-4096-9769-b1b9e4da5100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:40.008656 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3de96c-a34a-4096-9769-b1b9e4da5100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:40.010101 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3de96c-a34a-4096-9769-b1b9e4da5100 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:40.019485 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1436/2868] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:44.278484 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957365a9-4989-436d-afaf-0c2717153971 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:44.280750 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-957365a9-4989-436d-afaf-0c2717153971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:44.294390 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957365a9-4989-436d-afaf-0c2717153971 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:44.299074 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d149b-c8f2-499e-b4bb-15934d187217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:44.299733 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d149b-c8f2-499e-b4bb-15934d187217 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:44.300015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d149b-c8f2-499e-b4bb-15934d187217 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:44.300326 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d149b-c8f2-499e-b4bb-15934d187217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:44.309736 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d149b-c8f2-499e-b4bb-15934d187217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:44.311186 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d149b-c8f2-499e-b4bb-15934d187217 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:44.320955 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1433/2869] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:49.234343 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3de96c-a34a-4096-9769-b1b9e4da5100 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:49.300923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3de96c-a34a-4096-9769-b1b9e4da5100 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:49.306434 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:49.307311 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:49.307585 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:49.307894 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] PATH_INFO: `/v3/users/6cc7cd21122b4dae92178044851ae907` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:49.309274 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cc7cd21122b4dae92178044851ae907)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:49.311223 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:49.316541 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:49.363498 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cc7cd21122b4dae92178044851ae907'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:49.370919 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cc7cd21122b4dae92178044851ae907'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:49.371669 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cc7cd21122b4dae92178044851ae907'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:49.382717 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cc7cd21122b4dae92178044851ae907'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:49.398144 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cc7cd21122b4dae92178044851ae907'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:49.404615 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1437/2870] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:15:49 2022] DELETE /identity/v3/users/6cc7cd21122b4dae92178044851ae907 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:49.409527 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322820-6fa1-477b-9c86-0c5bd010ea07 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:49.410366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322820-6fa1-477b-9c86-0c5bd010ea07 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:49.410366 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322820-6fa1-477b-9c86-0c5bd010ea07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:49.428218 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b6322820-6fa1-477b-9c86-0c5bd010ea07 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:49.429446 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b6322820-6fa1-477b-9c86-0c5bd010ea07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:49.489020 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1434/2871] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:49.501967 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:49.504486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:49.570115 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246faab-e3e4-406f-9758-43b71a8f82eb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:49.574941 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6822546-1d66-4754-9b95-f22fee391185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:49.575597 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6822546-1d66-4754-9b95-f22fee391185 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:49.575874 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6822546-1d66-4754-9b95-f22fee391185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:49.576133 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6822546-1d66-4754-9b95-f22fee391185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:49.586975 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6822546-1d66-4754-9b95-f22fee391185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:49.588487 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6822546-1d66-4754-9b95-f22fee391185 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:49.598655 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1438/2872] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:49.881525 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6322820-6fa1-477b-9c86-0c5bd010ea07 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:49.883004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b6322820-6fa1-477b-9c86-0c5bd010ea07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:49.892195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:49.896745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6322820-6fa1-477b-9c86-0c5bd010ea07 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:49.902679 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:49.903636 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:49.903985 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:49.904336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] PATH_INFO: `/v3/projects/c6943bf93cda4888bad2dfd664681d0c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:49.905531 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6943bf93cda4888bad2dfd664681d0c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:49.907674 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:49.957134 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6943bf93cda4888bad2dfd664681d0c', 'request_id': 'req-8c236376-e1e6-48a3-812f-8f8b03093fa6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:49.963143 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6943bf93cda4888bad2dfd664681d0c', 'request_id': 'req-8c236376-e1e6-48a3-812f-8f8b03093fa6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:49.971705 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6943bf93cda4888bad2dfd664681d0c', 'request_id': 'req-8c236376-e1e6-48a3-812f-8f8b03093fa6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:49.982244 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] The token cache is being invalidate because project c6943bf93cda4888bad2dfd664681d0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:15:49.982524 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:49.983149 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:49.984756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1435/2873] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:15:49 2022] DELETE /identity/v3/projects/c6943bf93cda4888bad2dfd664681d0c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:49.988992 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6822546-1d66-4754-9b95-f22fee391185 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:49.999010 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6822546-1d66-4754-9b95-f22fee391185 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:50.059472 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:50.060373 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:50.060634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:50.060907 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] PATH_INFO: `/v3/users/6dbbc86a10014ad49aef4d0971c09c32` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:50.062106 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dbbc86a10014ad49aef4d0971c09c32)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:50.064038 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:50.094561 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:50.113430 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dbbc86a10014ad49aef4d0971c09c32'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:50.120270 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dbbc86a10014ad49aef4d0971c09c32'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:50.120928 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dbbc86a10014ad49aef4d0971c09c32'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:50.130384 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dbbc86a10014ad49aef4d0971c09c32'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:50.140769 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f46e5028-6c1f-41fc-b157-434086410dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dbbc86a10014ad49aef4d0971c09c32'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:50.146648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1439/2874] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:15:49 2022] DELETE /identity/v3/users/6dbbc86a10014ad49aef4d0971c09c32 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:50.437608 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:50.439179 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:50.452746 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c236376-e1e6-48a3-812f-8f8b03093fa6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:50.515532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:50.516362 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:50.516646 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:50.516923 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] PATH_INFO: `/v3/projects/f3a461ea782148d49f6b3afded195920` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:50.518252 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a461ea782148d49f6b3afded195920)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:50.520278 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:50.569725 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a461ea782148d49f6b3afded195920', 'request_id': 'req-70ac9cfa-17e5-4844-9d56-67e63de735cf'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:50.575715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a461ea782148d49f6b3afded195920', 'request_id': 'req-70ac9cfa-17e5-4844-9d56-67e63de735cf'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:50.585547 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a461ea782148d49f6b3afded195920', 'request_id': 'req-70ac9cfa-17e5-4844-9d56-67e63de735cf'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:50.595794 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] The token cache is being invalidate because project f3a461ea782148d49f6b3afded195920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:15:50.596087 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:50.596734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70ac9cfa-17e5-4844-9d56-67e63de735cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:15:50.598558 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1436/2875] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:15:50 2022] DELETE /identity/v3/projects/f3a461ea782148d49f6b3afded195920 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 21:15:50.604663 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213778fa-d99e-4497-95a8-c20f6c8803bf None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:50.605166 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213778fa-d99e-4497-95a8-c20f6c8803bf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:50.605282 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213778fa-d99e-4497-95a8-c20f6c8803bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:50.623225 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-213778fa-d99e-4497-95a8-c20f6c8803bf None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:50.624480 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-213778fa-d99e-4497-95a8-c20f6c8803bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:50.685371 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1440/2876] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:50.696373 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e027f-13da-46a5-9a7f-4d33690f0fa5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:50.696665 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e027f-13da-46a5-9a7f-4d33690f0fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:50.696943 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e027f-13da-46a5-9a7f-4d33690f0fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:50.718611 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-537e027f-13da-46a5-9a7f-4d33690f0fa5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:50.720004 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-537e027f-13da-46a5-9a7f-4d33690f0fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:50.730105 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1437/2877] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:50.734972 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213778fa-d99e-4497-95a8-c20f6c8803bf None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:50.749117 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213778fa-d99e-4497-95a8-c20f6c8803bf None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:50.755966 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28feb47-1e64-4040-8514-09118d4a6433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:50.756664 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28feb47-1e64-4040-8514-09118d4a6433 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:50.756939 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28feb47-1e64-4040-8514-09118d4a6433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:50.757212 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28feb47-1e64-4040-8514-09118d4a6433 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:50.757949 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28feb47-1e64-4040-8514-09118d4a6433 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:15:50.758256 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28feb47-1e64-4040-8514-09118d4a6433 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:50.759899 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28feb47-1e64-4040-8514-09118d4a6433 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:50.773770 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1441/2878] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:15:50.778521 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e027f-13da-46a5-9a7f-4d33690f0fa5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:50.788838 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e027f-13da-46a5-9a7f-4d33690f0fa5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:50.797969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789ee7b-98de-49d2-8ae5-43bebf8f8b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:50.798912 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789ee7b-98de-49d2-8ae5-43bebf8f8b27 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:50.799195 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789ee7b-98de-49d2-8ae5-43bebf8f8b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:50.799458 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789ee7b-98de-49d2-8ae5-43bebf8f8b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:50.800496 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789ee7b-98de-49d2-8ae5-43bebf8f8b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:50.803604 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789ee7b-98de-49d2-8ae5-43bebf8f8b27 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:50.815488 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:50.844402 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1438/2879] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:15:50 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:50.850109 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28feb47-1e64-4040-8514-09118d4a6433 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:50.861113 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28feb47-1e64-4040-8514-09118d4a6433 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:50.921360 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba0ab7-8111-4f7e-8e51-9c7ba7de0fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:50.922005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba0ab7-8111-4f7e-8e51-9c7ba7de0fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:50.922279 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba0ab7-8111-4f7e-8e51-9c7ba7de0fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:50.922556 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba0ab7-8111-4f7e-8e51-9c7ba7de0fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:50.923298 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba0ab7-8111-4f7e-8e51-9c7ba7de0fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:50.925263 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba0ab7-8111-4f7e-8e51-9c7ba7de0fc5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:50.947828 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1442/2880] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:15:50 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:15:50.952393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789ee7b-98de-49d2-8ae5-43bebf8f8b27 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:50.963407 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789ee7b-98de-49d2-8ae5-43bebf8f8b27 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:50.969296 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197720e6-4dd9-457c-a283-74de949174cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:50.970015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197720e6-4dd9-457c-a283-74de949174cc admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:50.970293 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197720e6-4dd9-457c-a283-74de949174cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:50.970570 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197720e6-4dd9-457c-a283-74de949174cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:50.971311 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197720e6-4dd9-457c-a283-74de949174cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:50.973454 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197720e6-4dd9-457c-a283-74de949174cc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:50.984678 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1439/2881] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 21:15:50.989018 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba0ab7-8111-4f7e-8e51-9c7ba7de0fc5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:50.999603 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba0ab7-8111-4f7e-8e51-9c7ba7de0fc5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:51.006544 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e26c8-8797-436b-8cd8-40222c8549a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:51.007521 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e26c8-8797-436b-8cd8-40222c8549a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:51.007851 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e26c8-8797-436b-8cd8-40222c8549a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:51.008273 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e26c8-8797-436b-8cd8-40222c8549a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:51.009375 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e26c8-8797-436b-8cd8-40222c8549a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:51.011757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e26c8-8797-436b-8cd8-40222c8549a5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:51.024778 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1443/2882] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:15:51.029153 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197720e6-4dd9-457c-a283-74de949174cc admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:51.039580 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197720e6-4dd9-457c-a283-74de949174cc admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:51.045668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c3b70-b52d-4772-be14-cb67d57def69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:51.046567 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3b70-b52d-4772-be14-cb67d57def69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:51.046845 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3b70-b52d-4772-be14-cb67d57def69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:51.047115 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3b70-b52d-4772-be14-cb67d57def69 admin admin] PATH_INFO: `/v3/projects/d39938fdeeb6442184b2bf75c30d87af/users/2e1ccae2b204463889e562d2ea486df1/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:51.062517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c3b70-b52d-4772-be14-cb67d57def69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39938fdeeb6442184b2bf75c30d87af, user_id=2e1ccae2b204463889e562d2ea486df1, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:51.065279 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c3b70-b52d-4772-be14-cb67d57def69 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:51.080653 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1440/2883] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:15:51 2022] PUT /identity/v3/projects/d39938fdeeb6442184b2bf75c30d87af/users/2e1ccae2b204463889e562d2ea486df1/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:15:51.092428 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ad771-a83e-4541-acac-459b098a8da0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:51.092723 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ad771-a83e-4541-acac-459b098a8da0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:51.093000 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ad771-a83e-4541-acac-459b098a8da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:51.112614 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:15:51.125961 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a35ad771-a83e-4541-acac-459b098a8da0 None None] MFA Rules not processed for user `2e1ccae2b204463889e562d2ea486df1`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:15:51.128254 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a35ad771-a83e-4541-acac-459b098a8da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:51.201598 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1444/2884] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:15:51.216013 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c3b70-b52d-4772-be14-cb67d57def69 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:51.218615 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-062c3b70-b52d-4772-be14-cb67d57def69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:15:51.296979 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c3b70-b52d-4772-be14-cb67d57def69 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:51.301494 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a194f3-2130-4993-af47-cb2bb4c66103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:51.302150 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a194f3-2130-4993-af47-cb2bb4c66103 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:51.302436 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a194f3-2130-4993-af47-cb2bb4c66103 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:51.302710 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a194f3-2130-4993-af47-cb2bb4c66103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:51.311455 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a194f3-2130-4993-af47-cb2bb4c66103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:51.313045 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a194f3-2130-4993-af47-cb2bb4c66103 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:15:51.322285 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1441/2885] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:15:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:15:59.903616 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ad771-a83e-4541-acac-459b098a8da0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:15:59.968767 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ad771-a83e-4541-acac-459b098a8da0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:15:59.973774 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:15:59.974648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:15:59.974923 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:15:59.975194 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] PATH_INFO: `/v3/users/2e1ccae2b204463889e562d2ea486df1` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:15:59.976478 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e1ccae2b204463889e562d2ea486df1)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:15:59.978412 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:00.028246 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e1ccae2b204463889e562d2ea486df1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:00.035462 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e1ccae2b204463889e562d2ea486df1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:00.036226 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e1ccae2b204463889e562d2ea486df1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:00.046460 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e1ccae2b204463889e562d2ea486df1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:00.059954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e1ccae2b204463889e562d2ea486df1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:00.066064 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1445/2886] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:15:59 2022] DELETE /identity/v3/users/2e1ccae2b204463889e562d2ea486df1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:16:00.070756 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1d081-60ab-4183-b920-47e4a7e4c89a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:00.071297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1d081-60ab-4183-b920-47e4a7e4c89a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:00.071297 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1d081-60ab-4183-b920-47e4a7e4c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:00.089005 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0fc1d081-60ab-4183-b920-47e4a7e4c89a None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:16:00.090184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0fc1d081-60ab-4183-b920-47e4a7e4c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:00.149668 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1442/2887] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:16:00.159852 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:00.161575 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:00.185103 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:16:00.228648 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237f127-8643-469e-8499-e35d8dff1d0b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:00.233396 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1e453-e781-44e4-aaa0-8c0455e02c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:00.234031 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1e453-e781-44e4-aaa0-8c0455e02c8a service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:00.234320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1e453-e781-44e4-aaa0-8c0455e02c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:00.234562 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1e453-e781-44e4-aaa0-8c0455e02c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:00.245099 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1e453-e781-44e4-aaa0-8c0455e02c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:00.246553 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1e453-e781-44e4-aaa0-8c0455e02c8a service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:00.256642 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1446/2888] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:16:00.537442 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1d081-60ab-4183-b920-47e4a7e4c89a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:00.539842 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0fc1d081-60ab-4183-b920-47e4a7e4c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:00.557396 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1d081-60ab-4183-b920-47e4a7e4c89a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:00.563910 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:00.564812 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:00.565124 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:00.565333 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] PATH_INFO: `/v3/projects/d39938fdeeb6442184b2bf75c30d87af` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:00.566616 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d39938fdeeb6442184b2bf75c30d87af)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:00.568638 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:00.620393 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd39938fdeeb6442184b2bf75c30d87af', 'request_id': 'req-46c95e7a-ead0-4477-ada4-285f16eef2a2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:00.622591 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:16:00.627727 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd39938fdeeb6442184b2bf75c30d87af', 'request_id': 'req-46c95e7a-ead0-4477-ada4-285f16eef2a2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:00.637184 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd39938fdeeb6442184b2bf75c30d87af', 'request_id': 'req-46c95e7a-ead0-4477-ada4-285f16eef2a2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:00.647745 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] The token cache is being invalidate because project d39938fdeeb6442184b2bf75c30d87af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:16:00.648032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:00.648696 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-46c95e7a-ead0-4477-ada4-285f16eef2a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:00.650275 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1443/2889] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:16:00 2022] DELETE /identity/v3/projects/d39938fdeeb6442184b2bf75c30d87af => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:16:00.655383 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd3749-6532-46f8-9f94-606152ba3444 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:00.655736 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd3749-6532-46f8-9f94-606152ba3444 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:00.656126 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd3749-6532-46f8-9f94-606152ba3444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:00.674813 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-98cd3749-6532-46f8-9f94-606152ba3444 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:16:00.676005 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-98cd3749-6532-46f8-9f94-606152ba3444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:00.738417 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1447/2890] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:16:00.747445 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f24971-7519-42e9-afce-a8238b64ffb2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:00.747735 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f24971-7519-42e9-afce-a8238b64ffb2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:00.748011 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f24971-7519-42e9-afce-a8238b64ffb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:00.767715 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e0f24971-7519-42e9-afce-a8238b64ffb2 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:16:00.768935 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e0f24971-7519-42e9-afce-a8238b64ffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:00.777652 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1444/2891] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:16:00.782266 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd3749-6532-46f8-9f94-606152ba3444 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:00.792998 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd3749-6532-46f8-9f94-606152ba3444 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:00.798915 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5df34-7f59-42f1-8253-fa00b93b9c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:00.799576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5df34-7f59-42f1-8253-fa00b93b9c7e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:00.799849 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5df34-7f59-42f1-8253-fa00b93b9c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:00.800158 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5df34-7f59-42f1-8253-fa00b93b9c7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:00.800887 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5df34-7f59-42f1-8253-fa00b93b9c7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 21:16:00.801193 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5df34-7f59-42f1-8253-fa00b93b9c7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:00.802801 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5df34-7f59-42f1-8253-fa00b93b9c7e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:00.816657 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1448/2892] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jul 11 21:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 21:16:00.821304 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f24971-7519-42e9-afce-a8238b64ffb2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:00.835100 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f24971-7519-42e9-afce-a8238b64ffb2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:00.842869 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefea5e-f469-403f-a738-7cc4eae6d1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:00.843532 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefea5e-f469-403f-a738-7cc4eae6d1ee admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:00.843809 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefea5e-f469-403f-a738-7cc4eae6d1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:00.844080 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefea5e-f469-403f-a738-7cc4eae6d1ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:00.844802 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefea5e-f469-403f-a738-7cc4eae6d1ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:00.846729 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefea5e-f469-403f-a738-7cc4eae6d1ee admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:00.875683 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1445/2893] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jul 11 21:16:00 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:16:00.879892 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5df34-7f59-42f1-8253-fa00b93b9c7e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:00.890088 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5df34-7f59-42f1-8253-fa00b93b9c7e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:00.947372 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba06e25-c946-4878-8f4b-ca18829dac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:00.948025 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba06e25-c946-4878-8f4b-ca18829dac69 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:00.948477 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba06e25-c946-4878-8f4b-ca18829dac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:00.948750 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba06e25-c946-4878-8f4b-ca18829dac69 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:00.949757 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba06e25-c946-4878-8f4b-ca18829dac69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:00.951482 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba06e25-c946-4878-8f4b-ca18829dac69 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:00.972683 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1449/2894] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jul 11 21:16:00 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 21:16:00.976858 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefea5e-f469-403f-a738-7cc4eae6d1ee admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:00.987675 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefea5e-f469-403f-a738-7cc4eae6d1ee admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:00.993232 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344a9fc-ec3b-48fa-bfb0-60b8b1146fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:00.993922 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344a9fc-ec3b-48fa-bfb0-60b8b1146fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:00.994229 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344a9fc-ec3b-48fa-bfb0-60b8b1146fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:00.994503 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344a9fc-ec3b-48fa-bfb0-60b8b1146fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:00.995235 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344a9fc-ec3b-48fa-bfb0-60b8b1146fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:00.997035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344a9fc-ec3b-48fa-bfb0-60b8b1146fb3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:01.007738 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1446/2895] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:16:01.011714 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba06e25-c946-4878-8f4b-ca18829dac69 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:01.021856 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba06e25-c946-4878-8f4b-ca18829dac69 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:01.027032 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9796c6-ee40-4763-ac64-8d1fafe40c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:01.027785 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9796c6-ee40-4763-ac64-8d1fafe40c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:01.028002 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9796c6-ee40-4763-ac64-8d1fafe40c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:01.028320 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9796c6-ee40-4763-ac64-8d1fafe40c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:01.029048 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9796c6-ee40-4763-ac64-8d1fafe40c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:01.030613 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9796c6-ee40-4763-ac64-8d1fafe40c1f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:01.040581 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1450/2896] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jul 11 21:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 21:16:01.044932 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344a9fc-ec3b-48fa-bfb0-60b8b1146fb3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:01.055654 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344a9fc-ec3b-48fa-bfb0-60b8b1146fb3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:01.061566 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60435cf-c9a5-4a45-970c-9324a4806e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:01.062448 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60435cf-c9a5-4a45-970c-9324a4806e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:01.062732 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60435cf-c9a5-4a45-970c-9324a4806e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:01.063000 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60435cf-c9a5-4a45-970c-9324a4806e85 admin admin] PATH_INFO: `/v3/projects/84066eca97da434bba7eccb134cbc251/users/451e35ee47624180926ba9eeebf664fb/roles/1403cb7e4e264c68b63e467a2e433699` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:01.079847 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60435cf-c9a5-4a45-970c-9324a4806e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84066eca97da434bba7eccb134cbc251, user_id=451e35ee47624180926ba9eeebf664fb, role_id=1403cb7e4e264c68b63e467a2e433699)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:01.082568 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60435cf-c9a5-4a45-970c-9324a4806e85 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:01.099734 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1447/2897] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jul 11 21:16:01 2022] PUT /identity/v3/projects/84066eca97da434bba7eccb134cbc251/users/451e35ee47624180926ba9eeebf664fb/roles/1403cb7e4e264c68b63e467a2e433699 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:16:01.115968 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c643c2-44c7-4e5f-ac68-9147cda68943 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:01.116431 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c643c2-44c7-4e5f-ac68-9147cda68943 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:01.116836 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c643c2-44c7-4e5f-ac68-9147cda68943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:01.149072 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-61c643c2-44c7-4e5f-ac68-9147cda68943 None None] MFA Rules not processed for user `451e35ee47624180926ba9eeebf664fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:16:01.150314 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-61c643c2-44c7-4e5f-ac68-9147cda68943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:01.168336 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:16:01.232089 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1451/2898] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:16:01.243347 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60435cf-c9a5-4a45-970c-9324a4806e85 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:01.245581 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b60435cf-c9a5-4a45-970c-9324a4806e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:01.305477 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60435cf-c9a5-4a45-970c-9324a4806e85 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:01.309673 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57163975-6947-48c2-aa1c-53c12806b2f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee928cd256446cb9e000ceb6e9ef636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:01.310321 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57163975-6947-48c2-aa1c-53c12806b2f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:01.310601 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57163975-6947-48c2-aa1c-53c12806b2f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:01.310881 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57163975-6947-48c2-aa1c-53c12806b2f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:01.320147 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57163975-6947-48c2-aa1c-53c12806b2f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:01.321626 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57163975-6947-48c2-aa1c-53c12806b2f8 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:01.330589 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1448/2899] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:16:11.228704 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=19 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:16:11.326782 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=58,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:16:11.503423 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c643c2-44c7-4e5f-ac68-9147cda68943 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:11.569679 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c643c2-44c7-4e5f-ac68-9147cda68943 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:11.574833 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:11.575715 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:11.576009 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:11.576291 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] PATH_INFO: `/v3/users/451e35ee47624180926ba9eeebf664fb` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:11.577524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] RBAC: Authorizing `identity:delete_user(user_id=451e35ee47624180926ba9eeebf664fb)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:11.579524 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:11.629954 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451e35ee47624180926ba9eeebf664fb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:11.637407 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451e35ee47624180926ba9eeebf664fb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:11.638176 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451e35ee47624180926ba9eeebf664fb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:11.646629 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451e35ee47624180926ba9eeebf664fb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:11.659470 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451e35ee47624180926ba9eeebf664fb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:11.665467 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1452/2900] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jul 11 21:16:11 2022] DELETE /identity/v3/users/451e35ee47624180926ba9eeebf664fb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:16:11.670517 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a84166-bbf6-41d1-ba92-409cd74b38f5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:11.670803 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a84166-bbf6-41d1-ba92-409cd74b38f5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:11.671076 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a84166-bbf6-41d1-ba92-409cd74b38f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:11.688597 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-65a84166-bbf6-41d1-ba92-409cd74b38f5 None None] MFA Rules not processed for user `5fc8fd4688454d8c8596f77d99812999`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 21:16:11.689964 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-65a84166-bbf6-41d1-ba92-409cd74b38f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:11.753984 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1449/2901] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jul 11 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 21:16:11.765755 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:11.768047 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:11.833701 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f960a55-d4d7-4987-b18e-edfa3ec2787f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:11.837883 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32e1b4-13ce-4b5b-be8e-cef61050b227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0574d5c8974f43218653208ca72c43bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:11.838576 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32e1b4-13ce-4b5b-be8e-cef61050b227 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:11.838858 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32e1b4-13ce-4b5b-be8e-cef61050b227 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:11.839136 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32e1b4-13ce-4b5b-be8e-cef61050b227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:11.849414 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32e1b4-13ce-4b5b-be8e-cef61050b227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:11.850873 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32e1b4-13ce-4b5b-be8e-cef61050b227 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:11.860823 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1453/2902] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:16:12.157035 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a84166-bbf6-41d1-ba92-409cd74b38f5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:12.158336 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-65a84166-bbf6-41d1-ba92-409cd74b38f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:12.171485 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a84166-bbf6-41d1-ba92-409cd74b38f5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:12.176486 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc8fd4688454d8c8596f77d99812999', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27db080494394cc5b886636221f8edb9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:12.178015 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:12.178502 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:12.178776 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] PATH_INFO: `/v3/projects/84066eca97da434bba7eccb134cbc251` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:12.179982 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84066eca97da434bba7eccb134cbc251)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:12.182032 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:12.231070 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84066eca97da434bba7eccb134cbc251', 'request_id': 'req-cce18b7b-cd06-4f38-ace9-b80c3a141646'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:12.237325 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84066eca97da434bba7eccb134cbc251', 'request_id': 'req-cce18b7b-cd06-4f38-ace9-b80c3a141646'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:12.245723 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84066eca97da434bba7eccb134cbc251', 'request_id': 'req-cce18b7b-cd06-4f38-ace9-b80c3a141646'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:12.256590 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] The token cache is being invalidate because project 84066eca97da434bba7eccb134cbc251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 21:16:12.256817 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:12.257430 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cce18b7b-cd06-4f38-ace9-b80c3a141646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 21:16:12.258969 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1450/2903] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jul 11 21:16:12 2022] DELETE /identity/v3/projects/84066eca97da434bba7eccb134cbc251 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 21:16:21.856388 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:16:22.253860 dv-cinder-849022-2-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 21:16:38.829486 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32e1b4-13ce-4b5b-be8e-cef61050b227 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 21:16:38.831316 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-db32e1b4-13ce-4b5b-be8e-cef61050b227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:38.939001 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32e1b4-13ce-4b5b-be8e-cef61050b227 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 21:16:39.004419 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae2fbb-2a1a-4961-b412-727581770b67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '946d11b552754364b55eb495f68ab852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc0dee82ab60459fa58fc3fcffc09043', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 21:16:39.005082 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae2fbb-2a1a-4961-b412-727581770b67 service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 21:16:39.005363 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae2fbb-2a1a-4961-b412-727581770b67 service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 21:16:39.005634 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae2fbb-2a1a-4961-b412-727581770b67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 21:16:39.007622 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a2ae2fbb-2a1a-4961-b412-727581770b67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 21:16:39.019183 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae2fbb-2a1a-4961-b412-727581770b67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 21:16:39.020817 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae2fbb-2a1a-4961-b412-727581770b67 service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 21:16:39.031533 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1454/2904] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jul 11 21:16:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 21:16:49.026841 dv-cinder-849022-2-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=67 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}